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^This  report  describes  one  phase  of  an  on-going  effort  to  apply  decision 
aiding  technology  to  Naval  Air  ASW.  Previous  portions  of  the  effort  have' 


identified  key  decision-making  situations  in  Naval  Air  ASW  to  which  decision- 
aiding  is  applicable,  and  specified  the  decision-aiding  techniques  that  are 
most  appropriate  for  each  of  them.  This  report  considers  the  problem  of 
prioritizing  those  situations  for  actual  decision  aid  development,  and  initiates 
the  aid  design  process  for  those  situations  found  to  have  the  highest  priority.  - 
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Initial  attempts  at  prioritizing  the  decision  situations  pointed  out  the 
need  to  treat  priority  as  an  explicitly  multidimensional  measure,  and  to  Incor¬ 
porate  the  judgement  of  experienced  Air  ASW  personnel  into  the  prioritization 
procedure.  These  two  requirements  led  to  the  development  of  a  novel  prioritiza¬ 
tion  methodology  termed  Priority  Mapping.  The  technique  utilizes  non-metric 
Multi-Dimensional  Scaling  (MDS)  and  Coombsian  Unfolding  Analysis  to  create 
quantitative  priority  scores  for  decision  situations  from  qualitative  judgement 
about  them  by  experienced  Air  ASW  personnel.  MDS  is  first  used  to  identify  the 
underlying  dimensions  with  which  experienced  personnel  perceive  ASW  decision¬ 
making.  Unfolding  Analysis  is  then  used  to  model  priority  in  term  of  these 
underlying  dimensions;  the  unfolding  analysis  model  provides  the  desired  numeric.! 
priority  scores. 

Two  types  of  data  for  the  prioritization  are  collected  from  32  highly 
experienced  P-3C  Tactical  Coordinators  (TACCOs)  at  NAS  Moffett  Field:  1)  judge¬ 
ments  about  the  basic  similarities/differences  among  a  sample  of  TACCO  decision 
functions,  and  2)  rankings  of  the  same  decision  functions  according  to  their 
perceived  importance  in  two  major  types  of  ASW  missions  —  attack  missions  and 
surveillance  missions.  MDS  Is  applied  to  the  first  type  of  data  (judged 
similarities)  to  dimensional ize  ASW  decision-making.  Three  dimensions  are 
identified  from  the  MDS  analysis:  1)  degree  of  uncertainty,  2)  information  pro¬ 
cessing  load,  and  3)  complexity  of  the  alternatives  involved.  Unfolding  Analysis 
is  applied  to  the  three  dimensional  MDS  solution  and  to  the  second  type  of  data 
(rankings)  to  create  a  mathematical  model  of  priority  as  a  function  of  the  three 
MDS  dimensions.  This  model  is  used  to  generate  numerical  priority  scores  for 
the  decision  functions  in  two  types  of  missions  (attack  and  surveillance)  which 
are  then  aggregated  and  normalized  to  construct  priority  scores  for  the  decision 
situations. 

For  both  attack  and  surveillance  missions,  the  Contact  Classification/Verif1 
cation  decision  situation  achieves  the  highest  priority.  The  Surveillance  Track' 
ing  situation  has  second  highest  priority  for  Surveillance  Missions,  while  the 
Attack  Planning  situation  has  second  highest  priority  for  Attack  Missions. 

Decision  aids  designs  for  two  of  the  highest  priority  situations— Contact 
Classification/Verification  and  Attack  Planning— are  initiated.  An  Optimal  Mode- 
Selection  decision  aid  is  outlined  for  the  Contact  Classification/Verification 
situation.  This  aid  uses  a  dynamic  programming  algorithm  and  a  multi  attribute 
utility  model  together  with  computational  algorithms  for  determining  the  proba¬ 
bility  of  correctly  Identifying  a  signal  from  a  particular  target  (given  a  signa' 
processing  mode  and  environmental  conditions)  to  help  the  TACCO  or  Sensor  Operati 
(SENSO)  select  an  optimal  processing  mode  for  his  acoustic  signal  processing 
equipment.  An  Attack  Criteria/Weapon  Placement  decision  aid  is  outlined  for  the 
Attack  Planning  decision  situation.  This  aid  uses  a  real-time  information 
processing  algorithm  to  automate  the  determination  of  attack  criteria  attainment 
and  uses  a  probabilistic  model  outcome  calculator,  a  nonlinear  value  model,  and 
a  nonlinear  programming  algorithm  to  optimize  the  tactics  for  anticipated  attack 


sccuhity  classification  or  this  nao«pwiw»  d««  *»*«•*> 


ACKNOWLEDGEMENTS 


The  research  presented  in  the  report  was  jointly  supported  by  the 
Engineering  Psychology  Program,  Office  of  Naval  Research  and  the  Human  Factors 
Division,  Naval  Air  Development  Center.  In  addition  to  the  author,  major  contri¬ 
butions  to  the  effort  were  made  by  Messrs.  James  Kelley,  Melvin  Strieb,  and 
Dr.  Robert  Wherry.  We  would  like  to  acknowledge  the  important  critical  contribu¬ 
tions  of  Dr.  Martin  Tolcott  and  Mr.  Gerald  Malecki  of  the  Office  of  Naval 
Research,  and  Dr.  Julie  Hopson,  LCDR.  Tomrny  Morrison,  CDR.  Michael  P.  Curran  and 
CDR.  Norman  Lane  of  the  Naval  Air  Development  Center.  Additional  thanks  are 
due  to  Mr.  Robert  Lawson  of  the  Office  of  Naval  Research  and  CDR.  David 
Stromberg  and  SQ.LDR.  Jon  Child  of  COMPATWINGSPAC  for  their  assistance  in 
arranging  the  interviews  at  Moffett  Field.  Last,  but  certainly  not  least,  we 
would  like  to  thank  the  TACCOs  from  squadrons  19,  31,  40,  46,  47,  and  50  at  NAS 
Moffett  Field  for  their  time  and  effort  in  providing  the  inputs  for  this 
research  that  only  they  could. 


i 


EXECUTIVE  SUMMARY 
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The  application  of  decision  aiding  technology  to  Naval  Air  ASW  requires 
a  broad  range  of  analytic  tools.  This  report  presents  the  results  of  the  current 
phase  of  Analytics'  effort  to  define  a  structure  for  applying  these  tools  within 
the  context  of  Naval  Air  ASW.  Previous  phases  of  this  effort  have  analyzed  Naval 
Air  ASW  platforms  and  missions,  identified  six  key  decision  making  situations 
which  arise  in  these  missions,  determined  the  decision  aiding  techniques  which 
are  best  suited  to  improving  decision  making  performance  in  each  of  them  and  made 
an  initial  attempt  at  decision  situation  prioritization.  The  selection  of  an 
appropriate  decision  situation  for  decision  aid  design  and  implementation  must  be 
based  upon  an  accurate  prioritization  of  the  decision  situations  according  to 
their  importance  to  the  Naval  Air  ASW  mission.  Our  initial  ASW  decision 
situation  prioritization  effort  pointed  out  the  need  to  treat  priority  as  a  multi¬ 
dimensional  measure,  and  the  need  to  incorporate  the  judgments  of  experienced 
operational  ASW  personnel  into  the  prioritization  procedure.  The  objective  of 
the  current  effort  reported  here  has  been  to  develop  and  apply  a  prioritization 
methodology  which  meets  these  requirements. 

A  prioritization  technique  called  Priority  Mapping  was  developed  in 
the  course  of  this  research.  This  technique  uses  Multidimensional  Scaling  and 
Unfolding  Analysis  to  translate  non-quantitative  judgements  about  the  similarity 
among,  and  relative  importance  of,  a  number  of  decision  functions  performed  by 
the  Tactical  Coordinator  (TACCO)  into  numerical  priority  scores  for  the  decision 
situations  in  which  these  decision  functions  arise. 

Two  types  of  basic  data  for  this  effort  were  collected  through  interviews 
with  32  highly  experienced  P-3C  TACCOs  stationed  at  NAS  Moffett  Field.  Data  on 
the  perceived  similarities  among  fourteen  decision  functions  were  gathered  and 
processed  to  produce  numerical  measures  of  their  dissimilarity.  These  data  were 
subsequently  input  to  a  Multidimensional  Scaling  program. 
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Additional  data  were  collected  on  the  perceived  importance  of  the  decision  func¬ 
tions  in  missions  with  and  without  attack  phases,  on  the  perceived  urgency  of 
the  decision  functions,  and  on  the  TACCOs1  perceptions  of  their  relative 
workload  during  each  of  the  decision  functions.  These  data  were  subsequently 
used  as  inputs  to  the  Unfolding  Analysis  procedure. 

A  Multidimensional  Scaling  analysis  of  the  perceived  similarities  data 
identified  the  ASW  decision  space  as  three  dimensional.  The  three  dimensions 
were  interpreted  as  representing: 

•  the  degree  of  uncertainty  in  the  information  used  to  make  a 
decision 

•  the  information  processing  load  a  decision  placed  on  the  TACCO 

•  the  complexity  of  the  alternatives  which  the  TACCO  considers 
in  making  the  decision. 

Unfolding  Analysis  was  conducted  to  assess  the  relative  ability  of  four 
models  of  priority  to  represent  the  preceived  importance  of  the  decision  func¬ 
tions  rated  by  the  TACCOs  at  Moffett  Field.  The  most  complex  model,  known  as 
the  generalized  distance  model,  was  selected  as  the  most  appropriate.  Two 
mathematical  formulae  which  represented  decision  priority  as  a  function  of  the 
three  dimensions  uncovered  by  the  Multidimensional  Scaling  procedure  were 
constructed  according  to  the  generalized  distance  model.  The  first  formula 
represented  the  decision  functions'  priority  in  attack  missions  while  the  second 
represented  their  priority  in  surveillance  missions.  The  resulting  decision 
function  priority  scores  were  aggregated  to  produce  priority  scores  for  the  six 
decision  situations  previously  identified.  For  attack  missions  the  prioritiza¬ 
tion  of  those  decision  situations  (in  order  of  descending  priority)  is: 

•  Contact  Classification/Verification 

•  Attack  Planning 

•  Localization 


•  Surveillance  Tracking 

•  Lost  Contact  Reacquisition 

•  On-Station  Search 

For  surveillance  missions,  the  prioritization  is: 

•  Contact  Classification/Verification 

•  Surveillance  Tracking 

•  Lost  Contact  Reacquisition 

•  Localization 

•  On-Station  Search 

•  Attack  Planning 

High  level  decision  aid  designs  are  presented  for  two  of  the  situations 
with  highest  priority  (Contact  Classification/Verification  and  Attack  Planning). 
An  Optimal  Mode-Selection  decision  aid  is  outlined  for  the  Contact 
Classification/Verification  situation.  This  aid  uses  a  dynamic  programming 
algorithm  and  a  multiattribute  utility  model  together  with  computational 
algorithms  for  determining  the  probability  of  correctly  identifying  a  signal 
from  a  particular  target  (given  specific  signal-processing  modes  and  environmen¬ 
tal  conditions)  to  help  the  TACCO  or  SENSO  select  an  optimal  processing  mode  for 
his  acoustical  signal  processing  equipment.  An  Attack  Criteria/Weapon  Placement 
decision  aid  is  outlined  for  the  Attack  Planning  decision  situation.  This  aid 
uses  a  real-time  information  processing  algorithm  to  automate  the  determination 
of  attack  criteria  attainment,  and  uses  a  probabilistic  model  outcome 
calculator,  a  nonlinear  value  model,  and  a  nonlinear  programming  algorithm  to 
optimize  the  tactics  for  anticipated  attack. 
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This  effort  has  resulted  in  three  primary  products: 


1.  Oimensionalization  of  ASW  decision-making  as  perceived 
by  experienced  ASW  decision  makers, 

2.  Prioritization  of  ASW  decision  situations  and  decision 
functions  in  two  kinds  of  ASW  missions,  and 

3.  Construction  of  high  level  decision  aid  designs  for 
two  of  the  decision  situations  with  highest  priority. 

Even  the  most  carefully  chosen  mix  of  decision  aiding  techniques  applied 
to  the  highest  priority  decision  situations,  however,  can  not  be  guaranteed  to 
improve  materially  the  quality  of  the  decision  made  in  that  situation.  The 
degree  of  uncertainty  Inherent  in  a  problem,  the  required  response  interval,  or 
other  such  limiting  factors  could  impose  such  severe  constraints  that,  in 
practice,  even  the  best  decision  aid  could  have  only  minimal  impact  on  given 
decision  situation.  It  still  remains  to  be  demonstrated  that  the  highest 
priority  decisions  are  in  fact  amenable  to  aiding  by  the  decision  aiding 
approaches  selected  here. 
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1.  INTRODUCTION  AND  METHODOLOGICAL  OVERVIEW 


The  rapid  increase  in  the  technological  sophistication  of  Naval  Air  ASW 
in  recent  years  has  brought  with  it  growing  demands  for  quicker  and  higher 
quality  decisions  from  ASW  aircrews.  This  report  describes  the  current 
Analytics  research  effort  in  investigating  the  application  of  decision  aiding  to 
Naval  Air  ASW.  The  project  reported  here  concerns  the  prioritization  of  Air  ASW 
decision  situations  for  decision  aiding  design. 

Analytics'  previous  investigation  into  Air  ASW  decision  aiding  (see 
Zachary,  1980)  has  focused  on  three  issues: 

•  the  definition  of  the  decision  points  within  the  generic 
Air  ASW  mission  to  which  decision  aiding  could  be  applied, 

•  the  identification  of  state-of-the-art  decision  aiding  techniques 
which  could  be  applied  to  each  of  these  decisions,  and 

•  the  prioritization  of  Air  ASW  decision  making  situations  according 
to  their  criticalities  in  the  overall  mission. 


These  issues  are  reviewed  below,  as  background  for  the  remainder  of  this  report. 


1.1  REVIEW  OF  PREVIOUS  RESEARCH 

An  analysis  of  the  commonalities  among  the  missions  flown  by  the  three 
principal  ASW  platforms  of  the  1980-1985  time  frame  —  P-3C,  and  S-3A,  and  the 
LAMPS  MK  III  —  has  resulted  in  the  construction  of  a  generic  Air  ASW  mission 
profile.  From  this  generic  mission  structure,  the  various  decisions  made  by  the 
Tactical  Coordinator  (TACCO)  nave  been  identified  and  investigated.  It  was 
found  that  similar  decision  processing  is  performed  by  the  TACCO  throughout  the 
mission,  but  with  different  end  results,  depending  on  the  particular  objective 
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of  the  mission  phase.  For  example,  the  goal  event  sought  by  the  TACCO  in  the 
on-station  search  portion  of  the  mission  is  gaining  contact  with  a  hostile 
submarine.  In  the  attack  planning  portion  of  the  mission,  the  goal  event  is  the 
destruction  of  the  submarine.  The  differing  goal  events  of  the  different 
mission  phases  give  rise  to  complex  decision  making  contexts,  which  constrain 
the  way  in  which  the  TACCO' s  primary  decision  functions  are  carried  out.  These 
contexts,  termed  decision  making  situations,  were  identified  as  the  principal 
units  to  which  decision  aiding  should  be  applied.  The  six  specific  decision 
situations  defined  are  listed  in  Table  1-1,  along  with  their  goal  events. 

Table  1-1.  ASW  Decision  Situations  and  Goal  Events 


DECISION  SITUATION 

GOAL  EVENT 

ON-STATION  SEARCH 

GAIN  CONTACT  WITH  TARGET  OF  INTEREST 

CONTACT  CLASSIFICATION/VERIFICATION 

IDENTIFY  SOURCE  OF  CONTACT 

LOCALIZATION 

DETERMINE  LOCATION.  COURSE.  SPEED  AND  DEPTH  OF  TARGET 

SURVEILLANCE  TRACKING 

MAINTAIN  LOCALIZED  CONTACT  WITH  TARGET 

ATTACK  PLANNING 

PLACE  OPTIMAL  ATTACK  AGAINST  HOSTILE  TARGET 

LOST  CONTACT  RE  ACQUISITION 

REGAIN  AND  LOCALIZE  CONTACT  WITH  A  LOST  TARGET 

Existing  decision  aids  were  reviewed  and  analyzed  to  determine  the 
range  and  characteristics  of  state-of-the-art  decision  aiding  technology. 
Complete  existing  decision  aids  were  found  to  be  highly  specialized,  and  not 
directly  generalizable  to  the  ASW  decision  situations.  However,  the  existing 
decision  aids  were  found  to  be  composed  of  similar  decision  aiding  techniques; 
these  individual  techniques  were  found  to  be  extremely  general.  Analysis  of  the 
functions  these  individual  techniques  played  in  the  various  aids  showed  there 
were  many  fewer  functional  categories  of  techniques  than  techniques  themselves. 
These  categories  were  used  to  group  the  techniques  into  a  functional  taxonomy 
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of  decision  aiding  methods.  The  categories  in  the  taxonomy  (shown  below  as 
Table  5-1)  were  used  as  the  basis  of  a  descriptive  framework  for  the  six  deci¬ 
sion  situations.  This  framework  permitted  the  situations  to  be  described  analy¬ 
tically  so  the  relevant  decision  aiding  techniques  could  be  matched  with 
specific  aspects  of  each  decision  situation. 


Finally,  the  decision  situations  were  prioritized  according  to  their 
need  or  priority  for  decision  aid  design.  A  two  factor  scheme  was  developed  for 
prioritizing  the  situations  which  considered  the  relative  operator  workload 
during  each  situation  and  the  positional  importance  of  each  situation. 
(Positional  importance  refers  to  the  interdependencies  among  the  situations 
caused  by  the  sequential  nature  of  the  ASW  mission;  if  early  situations  are  not 
successful,  then  later  situations  may  never  even  occur.)  The  prioritization 
achieved  by  this  procedure  is  as  follows: 

•  Lost  Contact  Reacquisition 

•  Contact  Classification/Verification 

•  On-Station  Search 

•  Localization 

t  Surveillance  Tracking 

•  Attack  Planning 


In  constructing  this  priority  scale,  two  critical  problems  emerged. 
First,  it  became  clear  that  many  more  factors  potentially  contributed  to 
priority  then  the  two  used.  There  appeared  to  be  no  simple  way,  however,  to 
determine  which  factors  were  actually  relevant  to  priority  and  how  they  contri¬ 
buted  to  it.  Second,  the  prioritization  was  analytical  (rather  than 
experimental)  and  therefore  did  not  incorporate  the  knowledge,  judgements,  and 
intuition  about  ASW  decision  making  possessed  by  experienced  operational  ASW 
personnel.  The  importance  of  this  second  problem  became  clear  when  an  attempt 
was  made  to  validate  the  prioritization  by  comparing  it  with  rankings  produced 
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by  a  small  number  of  operational  ASW  personnel  at  the  Naval  Air  Development 
Center  (NADC).  Although  only  a  small  sample  of  ranking  data  was  collected  (i.e. 
N=4),  there  were  large  differences  between  the  individual  rankings  which  all 
differed  substantially  from  the  analytically  produced  rankings. 

Given  the  likelihood  that  significant  factors  had  been  excluded  from 
the  original  prioritization  and  given  the  inability  to  validate  it  with  opera¬ 
tional  personnel,  it  was  imperative  to  validate  the  prioritization  with  a  more 
sophisticated  methodology  that  would  overcome  the  two  problems  of  the  original 
analysis.  The  majority  of  the  effort  reported  here  was  devoted  to  this  task. 

1.2  METHODOLOGY  FOR  DECISION  SITUATION  PRIORITIZATION 

In  validating  the  previously  constructed  prioritization,  a  methodology 
was  sought  which  could  simultaneously  treat  the  multidimensional  nature  of  deci¬ 
sion  priority,  as  well  as  incorporate  the  knowledge,  judgements  and  intuitions 
of  experienced  ASW  personnel.  In  particular,  the  methodology  was  required  to 
provide  means  for 

•  identifying  the  factors  relevant  to  ASW  decision 
priority, 

•  determining  the  manner  in  which  these  factors  combined  to  yield 
priority,  and 

•  constructing  priority  scores  for  the  decision  situations. 

The  requirement  that  experienced  ASW  personnel  be  incorporated  into  the  analysis 
necessitated  a  psychometric  approach  to  the  problem.  After  many  standard  tech¬ 
niques  were  reviewed  and  judged  to  be  unsatisfactory,  a  new  prioritization 
methodology  was  developed  based  on  Multidimensional  Scaling. 

In  this  methodology,  the  underlying  dimensions  which  experienced  ASW 
personnel  find  relevant  to  ASW  decision-making  are  identified  through  a 
Multi-Dimensional  Scaling  (MDS)  analysis  of  basic  judgemental  data  supplied  by 
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these  personnel.  Next,  Unfolding  Analysis  is  used  to  determine  the  way  in  which 
these  dimensions  combine  to  produce  a  scale  of  priority  in  the  minds  of  these 
individuals.  Finally,  a  technique  known  as  Priority  Mapping  is  used  to 
construct  numerical  priority  functions  for  the  decision  situations  in  terms  of 
the  dimensions  in  the  MDS  solution.  This  overall  methodology  is  pictured  in 
Figure  1-1.  The  various  steps  in  this  methodology,  as  well  as  a  brief  review 
of  the  result  of  its  application  are  discussed  below. 

1.2.1  Collecting  the  Data  for  the  Prioritization 

The  Multidimensional  Scaling  and  Unfolding  Analysis  techniques  both 
require  as  inputs  psychometric  data  collected  from  individuals  highly  familiar 
with  the  items  being  scaled.  In  the  case  of  ASM  decision  making,  this  meant 
that  the  data  for  the  effort  had  to  be  collected  from  experienced  ASW  decision 
makers.  For  this  study,  an  "experienced"  individual  was  operationally  defined 
as  one  who  has  served  at  least  one  year  or  one  full  deployment  as  a  TACCO.  The 
largest  available  pool  of  individuals  with  this  level  of  TACCO  experience  was  at 
NAS  Moffett  Field,  so  the  data  needed  for  the  prioritization  procedure  were  collected 
there. 

Two  basic  types  of  data  were  collected  from  each  experienced  TACCO 
through  the  vehicle  of  psychometric  tasks.  The  first  task  performed  by  each 
individual  was  an  unconstrained,  sorting  of  a  sample  of  fourteen  decision 
functions:  a  grouping  of  TACCO  decision  functions  such  that  all  "similar"  deci¬ 
sions  are  grouped  together.  No  restrictions  were  placed  on  the  number  or  size 
of  groups  that  could  be  formed.  The  decision  functions  (individual  decisions 
made  by  a  TACCO  in  the  course  of  a  mission)  were  used  instead  of  the  six  deci¬ 
sion  situations  for  two  reasons: 

1)  The  original  prioritization  was  based  on  decision  function 
priority  scores  aggregated  across  decision  situations. 

This  required  the  decision  function  to  be  the  basic  unit 
of  the  present  prioritization  in  order  to  ensure  compara¬ 
bility  between  the  two  analyses. 


TACCOs 


Figure  1-1.  Multidimensional  Scaling  Prioritization  Procedure 
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2)  The  decision  situations  were  analytical  constructs  which 

were  not  likely  to  be  immediately  familiar  to  the  TACCOs  without 
considerable  explanation.  The  decision  functions  on  the 
other  hand,  were  directly  recognizable  by  the  TACCOs  because 
they  refer  to  specific  tasks  actually  performed  by  them. 

The  fourteen  decision  functions  selected  were  chosen  because  they  occurred  in 
several  of  the  decision  situations  and  because  they  represent  a  sample  of  all 
types  of  tactical  decisions  which  arise  in  the  course  of  a  mission. 

The  unconstrained  sorting  task  provides  basic  information  as  to  the 
perceived  similarities  among  the  decision  functions.  This  information  could  be 
preprocessed  and  used  as  input  to  an  MDS  program.  The  unconstrained  sorting 
task  was  selected  over  other  tasks  (such  as  paired-comparisons  or  triads) 
because  it  is  relatively  easy  to  apply  and  it  permits  a  powerful  mathematical 
analysis  (see  Section  2. 2. 2.1). 

The  second  task  performed  by  each  TACCO  was  a  series  of  rankings  of  the 
decision  functions  by  several  different  criteria.  The  criteria  were: 

•  importance  of  the  decision  function  in  a  mission  whose  objective 
is  to  attack  the  hostile  submarine, 

•  importance  of  the  decision  function  in  a  mission  whose  objective 
is  only  to  achieve  surveillance  on  the  hostile  submarine, 

•  urgency  of  the  decision  function,  and 

•  TACCO  workload  during  the  decision  function. 


The  ranking  task  provides  basic  information  on  the  perceived  relationships  among 
the  decision  functions  with  regard  to  each  of  these  criteria.  The  ranking 
results  are  used  as  input  to  the  Unfolding  Analysis. 


effort  to  develop  decision  aids  for  Naval  Air  ASW  and  was  followed  by  the  intro¬ 
duction  of  the  ranking  and  unconstrained  sorting  tasks.  The  interview  ended 
with  an  open  discussion  of  decision  aids  and  human  engineering  issues  in  ASW. 
Further  details  on  the  data  collection  procedures  for  this  effort  are  presented 
in  Section  2. 

1.2.2  Preprocessing  the  Unconstrained  Sorting  Data 

The  unconstrained  sorting  task  is  designed  to  capture,  in  a  non- 
quantitative  manner,  the  TACCO's  perception  of  the  interrelationships  among  the 
decision  functions  being  considered.  The  information  contained  in  the  sorting 
results  needs  to  be  represented  quantitatively  before  it  can  be  used  as  an  input 
to  a  Multidimensional  Scaling  program.  This  can  be  accomplished  with  a  computer 
program  named  METRIC  which  is  based  on  algorithm  developed  by  Burton  (1975).  From 
the  results  of  an  unconstrained  sorting,  METRIC  computes  an  indicator  or 
measure  of  the  dissimilarity  of  each  pair  of  decision  functions  used  in  the 
unconstrained  sorting.  The  measure  is  expressed  as  a  square  matrix  in  which 
each  decision  function  is  represented  by  one  row  and  column.  The  dissimilarty 
matrix  can  be  directly  input  to  an  MDS  program.  Since  fourteen  decision  func¬ 
tions  were  used  in  the  unconstrained  sorting  task,  the  dissimilarity  matrix 
which  results  from  the  application  of  METRIC  to  the  data  colllected  at  Moffett 
Field  contains  fourteen  columns  and  fourteen  rows.  This  preprocessing  step  is 
described  in  more  detail  in  Section  2.3. 

1.2.3  Multidimensional  Scaling  of  the  Decision  Functions 

The  groupings  of  the  decision  functions  formed  by  the  TACCOs  in  the 
unconstrained  sorting  task  reflect  the  underlying  distinctions  or  features  which 
these  experienced  TACCOs  use  to  organize  their  thinking  about  ASW  decision 
making.  Multidimensional  Scaling  is  a  technique  which  'reconstructs'  or  iden¬ 
tifies  these  underlying  features  by  analyzing  a  dissimilarity  matrix  created 
from  the  results  of  an  unconstrained  sorting.  MDS  does  this  by  defining  the 
objects  being  scaled  as  points  in  a  multidimensional  topological  space  and 
"building"  the  space  which  best  represents  the  objects  in  it.  The  quality  of 
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the  representation  is  determined  by  a  comparison  of  the  diatanoea  of  the 
objects  in  the  space  with  the  di88imilaritiea  of  the  objects  used  as  input  to 
the  procedure.  In  non-metric  MDS,  the  most  general  type  of  MDS,  the  fit  is  con¬ 
sidered  perfect  if  the  dissimilarities  are  a  monotonic  transformation  of  the 
distances.  Thus,  only  the  ordinal  properties  of  the  dissimilarity  matrix  are 
used  in  the  MDS  rather  than  the  actual  numerical  values  in  the  matrix.  This  is 
desirable  because  the  numbers  in  the  matrix  have  no  psychological  interpretation, 
and  should  not  be  assumed  to  represent  a  precise  ratio  measurement  scale  (see 
Section  3.1). 


M 


In  an  MDS  solution,  the  items  being  scaled  (here,  the  ASM  decision 
functions)  are  represented  as  points  in  some  multidimensional  space.  Each 
dimension  or  axis  in  this  space  represents  a  distinctive  feature  used  to 
organize  the  scaled  items  (called  stimuli)  in  the  minds  of  the  individuals  doing 
the  unconstrained  sorting.  The  meaning  of  each  dimension  is  then  interpreted  by 
examining  the  projections  of  each  of  the  stimuli  onto  it.  The  features  repre¬ 
sented  by  the  MDS  dimensions  are  the  only  ones  which  are  actually  used  by 
experienced  personnel  to  "think  about"  the  items  involved.  Therefore,  they 
should  be  the  only  factors  required  in  the  computation  of  a  scale  of  priority. 

The  dissimilarity  matrix  constructed  from  the  unconstrained  sortings 
gathered  at  Moffett  Field  was  input  into  a  Multidimensional  Scaling  program  named 
MINISSA.  The  program  identified  the  ASW  decision  space  as  three  dimensional. 
Plots  of  the  various  decision  functions  on  these  three  dimensions  are  given 
below  in  Figure  3-2.  These  dimensions  were  tentatively  interpreted  as  follows: 

•  Dimenaion  1  -  the  uncertainty  in  the  information  used 

to  make  a  decision, 

•  Dimenaion  2  -  the  information  processing  load  the 
decision  imposes  on  the  TACCQ,  and 

•  Dimenaion  3  -  the  complexity  of  the  alternatives 
the  TACCO  considers  in  making  the  decision. 


Further  details  on  the  W)S  analysis  are  given  below  in  Section  3. 
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1.2.4  Unfolding  Analysis  of  the  Ranking  Criteria 

The  MDS  analysis  identifies  the  underlying  dimensions  of  the  perceptual 
space  of  ASW  decision  making  from  the  results  of  the  TACCO's  unconstrained 
sortings.  Other  features  of  the  ASW  decision  functions,  such  as  their  perceived 
importance  values,  must  therefore  be  combinations  for  functions  of  these  three 
main  dimensions.  Unfolding  Analysis  is  a  technique  for  determining  the  specific 
form  of  the  function  which  best  represents  a  derivative  feature  (e.g.  importance) 
as  a  combination  of  the  independent  features  of  the  perceptual  space  (i.e.  the 
dimensions  of  the  MDS  solution).  In  Unfolding  Analysis,  the  ability  of  four 
different  models  of  dimensional  combination  to  represent  a  given  ranking  of  the 
stimuli  are  tested  and  compared.  The  model  which  is  simplest  and  which  has  the 
best  fit  with  the  ranking  is  then  selected  as  the  form  by  which  the  ranking  cri¬ 
terion  is  mapped  into  the  multidimensional  space.  A  general  discussion  of 
Unfolding  Analysis  is  given  in  Appendix  F. 

Two  of  the  rankings  collected  at  Moffett  Field  were  used  along  with  the 
three  dimensional  MIS  solution  in  the  Unfolding  Analysis.  The  ranking  criteria 
considered  were  those  relating  to  importance  (in  missions  with  an  attack  phase 
and  in  missions  without  an  attack  phase)  because  preceived  importance  was  con¬ 
sidered  equivalent  to  priority.  The  average  of  the  rankings  produced  for  each 
of  the  two  criteria  by  all  the  32  TACCO's  interviewed  were  used  as  the  input  to 
the  Unfolding  Analysis.  The  most  powerful  of  the  four  models,  called  the 
generalized  distance  model,  produced  the  best  fit  of  the  rankings  in  the  multi¬ 
dimensional  space  for  both  criteria,  accounting  for  the  99%  of  the  variance  in 
both  cases.  The  Unfolding  Analysis  procedure  is  summarized  in  Section  4.1; 
details  of  calculations  are  given  in  Appendix  G. 

1.2.5  Priority  Mapping  of  the  Decision  Functions  and  Decision  Situations 

The  analytical  form  of  the  function  which  expresses  the  criteria  of 

importance  in  missions  with  and  without  attack  in  terms  of  the  three  MDS  dimen¬ 
sions  is  determined  by  the  Unfolding  Analysis.  Next,  a  regress i on- 1  ike  proce¬ 
dure,  called  Priority  Mapping  is  used  to  specify  the  values  of  the  coefficients 
in  a  specific  priority  function  for  each  criterion.  The  result  of  Priority 

fAI 
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Mapping  is  a  complete  formula  which  can  generate  numerical  priority  scores  for 
the  decision  functions,  given  their  locations  on  each  of  the  dimensions  in  the 
MDS  solution.  The  procedure  is  regression- like  because  it  fits  coefficient  in 
an  analytic  function  to  ordinal  data  (the  rankings  on  each  criterion).  Rather 
than  using  a  standard  least-squares  estimation,  it  employs  a  two-step  iterative 
procedure.  It  first  computes  a  least-squares  estimation  using  the  ranks  and 
then  adjusts  the  results  to  fit  the  ordinal  properties  of  the  solution  back  to 
the  original  data.  The  two  steps  are  iterated  until  either  an  acceptable  fit  is 
achieved  or  it  becomes  clear  that  no  progress  toward  a  solution  is  being  made. 

A  program  named  PREFMAP  is  used  to  carry  out  the  Priority  Mapping  procedure  (see 
Section  4.1). 

Priority  Mapping  was  used  to  generate  representations  of  the  two  impor¬ 
tance  ranking  criteria  as  functions  of  the  three  MDS  dimensions  according  to  the 
generalized  distance  model.  The  priority  function  for  the  criterion  of  impor¬ 
tance  in  a  mission  with  attack  is  given  in  Appendix  G,  and  the  numerical  decision 
function  priority  scores  generated  by  this  formula  are  given  in  Table  4-1.  The 
priority  function  for  the  criterion  of  importance  in  a  mission  without  attack  is 
given  in  Appendix  G,  and  the  decision  function  priority  scores  generated  by  this 
formula  are  given  in  Table  4-2. 

The  decision  function  prioritizations  were  converted  into  decision- 
situation  prioritizations  by  summing  the  priority  scores  of  the  decision  func¬ 
tions  which  comprised  each  decision  situation.  These  "raw1'  decision  situation 
priority  scores  were  then  normalized  by  dividing  each  of  them  by  the  number  of 
decision  functions  summed  to  produce  it.  The  normalized  priority  scores  for 
the  decision  situations  were  used  as  the  final  priority  values.  The  final 
ranking  of  the  decision  situations  in  each  type  of  mission  are  shown  in  Table 
1-2.  A  complete  presentation  of  the  Priority  Mapping  analysis  is  given  in 
Section  4.2  and  Appendix  G. 


Table  1-2.  ASW  Decision  Situation  Prioritizations  in  Two  Types  of  ASW  Missions 


RANK  IN 
MISSION  WITH 
ATTACK 
OBJECTIVE 

DECISION 

SITUATION 

DECISION 

SITUATION 

RANK  IN 
MISSION 
SURVEILLANCE 
OBJECTIVE 

1 

CONTACT  CLASSIFICATION/ 

CONTACT  CLASSIFICATION/ 

1 

VERIFICATION 

VERIFICATION 

2 

ATTACK  PLANNING 

SURVEILLANCE  TRACKING 

2 

3 

LOCALIZATION 

LOST  CONTACT  REACQUISITION 

3 

4 

SURVEILLANCE  TRACKING 

LOCALIZATION 

4 

5 

LOST  CONTACT  REACQUISITION 

ON-STATION  SEARCH 

5 

6 

ON-STATION  SEARCH 

ATTACK  PLANNING 

6 

1.3  DECISION  AID  DESIGN 

The  prioritization  given  in  Table  1-2  indicates  the  order  in  which  the 
six  decision  situations  should  be  considered  for  decision  aiding  applications. 

As  an  initial  step  in  this  direction,  high  level  decision  aid  designs  were 
constructed  for  two  of  the  decision  situations.  For  the  Contact  Classification/ 
Verification  decision  situation  (which  ranked  first  in  both  types  of  mission)  an 
Optimal  Mode  Selection  decision  aid  is  outlined.  This  aid  helps  the  TACCO 
and/or  the  SENSO  (Sensor  Station  Operator)  select  the  optimal  combination  of 
processing  modes  for  the  acoustical  signal  processing  equipment  based  on  a  set 
of  targets-of-interest  and  the  in-situ  environmental  conditions.  For  the  Attack 
Planning  decision  situation  (which  ranked  second  for  missions  with  an  attack 
phase)  an  attack  criteria/attack  plan  decision  aid  is  outlined.  This  decision 
aid  helps  the  TACCO  in  attaining  attack  criteria  on  the  submarine  as  quickly  as 
possible  and,  once  they  are  obtained,  assists  him  in  selecting  and  implementing 
the  optimal  tactics  for  the  actual  attack.  The  full  outline  of  these  decision 
aids  are  presented  in  Section  5. 


1.4  ORGANIZATION  OF  THE  REPORT 

The  remainder  of  this  report  presents  the  results  of  the  prioritization 
analysis  and  decision  aid  design  in  greater  detail.  Section  2  discusses  the 
data  collection  procedures,  and  the  preliminary  analyses  of  the  data  gathered  at 
Moffett  Field.  Section  3  discusses  the  application  of  Multidimensional  Scaling 
to  dimensional ize  the  ASW  decision  space  using  the  data  collected  at  Moffett 
Field.  Section  4  discusses  the  application  of  the  Unfolding  Analysis  and 
Priority  Mapping  procedures  to  the  prioritization  of  the  six  Air  ASW  decision 
situations.  Section  5  presents  initial  high-level  decsion  aid  designs  for  two 
of  the  highest  priority  situations.  Section  6  presents  conclusions  and  recom¬ 
mendations  based  on  the  results  of  the  effort.  The  materials  used  in  the 
Moffett  Field  interviews  and  the  raw  data  gathered  from  them  are  presented  in 
the  appendices  to  this  report. 


2.  DATA  COLLECTION  FOR  THE  MULTIDIMENSIONAL  SCALING 
AND  UNFOLDING  ANALYSIS  PROCEDURES 

This  research  is  based  on  the  assumption  that  a  prioritization  of  Air 
ASW  decisions  must  incorporate  the  knowledge,  opinions,  and  intuitions  of 
experienced  air  ASW  decision-makers.  Multidimensional  scaling  and  unfolding 
analysis  provide  a  mechanism  for  numerically  prioritizing  the  six  decision 
situations  on  the  basis  of  structured  but  fundamentally  judgemental  inputs  from 
experienced  ASW  personnel.  This  section  describes  the  procedures  used  to  obtain 
and  preprocess  the  judgemental  data  needed  to  perform  the  MDS  and  unfolding 
analysis.  The  characteristics  of  the  personnel  from  whom  data  were  collected, 
and  the  choice  of  a  location  for  the  data  collection  procedure  are  discussed  in 
Section  2.1.  In  Section  2.2,  the  structure  of  the  interview  used  to  obtain  the 
data  is  described.  The  data  which  were  collected,  and  the  techniques  used  to 
preprocess  them  for  the  MDS  and  unfolding  analysis  are  reviewed  in  Section  2.3. 

2.1  AIR  ASW  PERSONNEL  REQUIREMENTS 

In  the  earlier  effort  (Zachary,  1980),  three  air  ASW  platforms  were  iden¬ 
tified  as  the  focus  of  the  investigation  --  P-3C,  S-3A,  and  LAMPS  MK  III.  Only 
the  first  two  of  these  are  currently  operational,  so  the  personnel  to  be  inter¬ 
viewed  were  restricted  to  those  currently  working  on  the  P-3C  and  the  S-3A.  The 
previous  effort  also  identified  the  Tactical  Coordinator  or  TACCO  as  the  princi¬ 
pal  decision-maker  of  interest,  further  restricting  the  population  to  individuals 
serving  as  TACCOs  on  these  two  platforms.  The  motivation  for  the  approach  taken 
in  the  effort,  as  well  as  the  underlying  assumptions  of  the  psychometric  proce¬ 
dures  used,  required  that  the  TACCOs  interviewed  be  highly  experienced  at  their 
job.  An  experienced  TACCO  was  operationally  defined  as  someone  who  had  served 
as  a  designated  or  a  crew-flying  (airborne)  TACCO  aboard  the  P-3C  or  S-3A  for  at 
least  one  year  or  one  full  deployment.  Prospective  interviewees,  then,  were 
defined  as  individuals  having  that  type  of  TACCO  experience. 
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Five  possible  sources  of  interviewees  within  the  continental  US  were 
identified.  Three  of  these  provided  P-3C  populations  (the  airwings  at 
Brunswick,  Maine,  Jacksonville,  Florida,  and  Moffett  Field,  California),  and  two 
provided  S-3A  populations  (the  airwings  at  Cecil  Field,  Florida  and  San  Diego, 
California).  The  scope  of  the  present  effort  allowed  interviews  to  be  conducted 
at  only  one  of  these  sites.  It  was  initially  decided  to  restrict  the  candidate 
sites  to  the  P-3C  locations,  to  accommodate  the  need  for  a  preliminary  or  pilot 
data  collection  procedure  (see  Section  2.3).  This  pilot  procedure  was  conducted 
to  allow  deficiencies  in  the  interview  method  to  be  identified  and  corrected 
prior  to  the  principal  data  collection  effort.  The  preliminary  procedure  was  to 
be  conducted  at  NADC  where  the  preponderance  of  available  interviewees  had  a 
P-3C,  rather  than  an  S-3A,  background. 

Of  the  three  possible  sources  of  P-3C  interviewees,  Brunswick  was  imme¬ 
diately  eliminated  because  their  transition  to  the  P-3C  (from  A  and  B)  was  suf¬ 
ficiently  recent  that  the  criteria  of  experience  could  not  be  met.  Of  the 
remaining  two,  Moffett  Field  was  chosen  because  it  had  available  the  largest  pool 
of  potential  interviewees  during  the  timeframe  in  which  the  interviews  were  to 
be  conducted. 

2.2  STRUCTURE  OF  THE  MULTIDIMENSIONAL  SCALING  INTERVIEW 

The  data  for  the  MDS  and  unfolding  analysis  were  collected  through  sev¬ 
eral  psychometric  tasks  presented  as  part  of  a  structured  interview.  The  format 
of  the  interview  allowed  several  TACCOs  to  be  interviewed  in  the  same  room  simul- 
taneouly.  The  overall  interview  had  three  parts.  Initially,  a  briefing  was  pro¬ 
vided  on  the  history  of  the  ASW  decision  aids  effort  and  its  goals.  Next,  a 
series  of  structured  psychometric  tasks  were  presented  to,  and  performed  by,  each 
interviewee.  Finally,  an  open-ended  discussion  among  the  interviewers  and  inter¬ 
viewees  concerning  decision  aids  and  general  human  factors  engineering  problems 
in  the  P-3C  was  held.  Each  of  these  portions  of  the  interview  is  discussed  in 


further  detail  below.  The  interviews  are  designed  to  be  completed  in  a  single 
morning  or  afternoon,  scheduled  as  follows: 


•  One  hour  was  allocated  to  initial  briefing  and 
questions-and-answers; 

•  One  half  hour  was  allocated  to  the  first  of  the  psychometric 
tasks  (unconstrained  sorting); 

•  Forty-five  minutes  was  allocated  to  the  second  of  the 
psychometric  tasks  (ranking);  and 

•  Two  hours  were  used  for  the  final  discussion  session. 

2.2.1  Initial  Briefing 

At  the  start  of  each  interview  session,  a  short  introductory  briefing 
was  given.  The  first  part  of  this  briefing  covered  the  work  and  data  on  this 
project,  emphasizing  the  notion  of  a  decision  aid,  the  interest  the  Navy  has  in 
decision  aids,  and  prior  analysis  of  a  generic  Air  ASW  mission  into  mission 
segments,  decision  functions,  and  decision  situations.  The  second  part  of  this 
briefing  covered  the  need  to  order  (prioritize)  these  decision  situations  as  a 
way  of  identifying  the  one(s)  to  which  decision-aiding  should  be  applied.  The 
need  and  desire  to  base  this  prioritization  analysis  on  the  experience  and 
knowledge  of  operational  personnel  was  emphasized. 

During  the  third  portion  of  the  briefing  the  subjects  were  introduced 
to  the  tasks  they  would  be  asked  to  perform.  They  were  assured  their  responses 
would  remain  anonymous  and  not  be  taken  as  a  statement  of  official  Naval  or  air¬ 
wing  policy.  The  full  text  of  the  briefing  is  given  in  Appendix  A.  Questions, 
comments,  and  other  spontaneous  discussion  were  then  solicited  and  responses 
provided  prior  to  the  beginning  of  the  first  of  the  psychometric  tasks. 

2.2.2  Psychometric  Tasks 

The  TACCOs  were  asked  to  perform  two  psychometric  tasks,  the  purpose  of 
which  was  to  provide  information  that  could  be  used  to  derive  numerical  indicators 
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of  various  subjective  aspects  of  their  ASW  knowledge.  The  first  task,  an 
unconstrained  sorting  task,  provided  the  inputs  needed  for  the  multidimensional 
scaling.  The  second  task,  a  simple  rank-ordering,  provided  the  inputs  to  the 
unfolding  analysis  and  was  performed  in  four  different  ways  as  described  in  the 
following  subsections. 

2. 2. 2.1  The  Unconstrained  Sorting  Task 

The  techniques  of  unconstrained  sorting,  also  known  as  the  "Q-sort," 
was  first  developed  by  George  Miller  (1969).  In  this  task,  the  subject  is  asked 
to  place  the  items  being  scaled  (called  collectively  the  stimuli)  into  an  arbi- 
rary  number  of  groups  of  unspecified  size.  The  stimuli  may  be  organized  into  as 
many  groups  as  is  necessary  to  place  all  similar  items  together,  without  any 
restrictions  on  how  many  items  may  be  placed  into  each  group.  The  basis  on 
which  the  items  are  grouped  is  established  by  the  instructions  for  the  task. 

The  unconstrained  sort  is  only  one  of  many  psychometric  methods  used  to 
collect  data  for  multidimensional  scaling.  Other  commonly  used  methods  are  the 
triads  test,  paired  comparisons,  and  conjoint  measurement.  Unconstrained 
sorting  was  chosen  over  these  other  techniques  for  two  reasons.  First,  it  is 
the  only  technique  which  allows  the  levels  of  distinction  drawn  among  the  stim¬ 
uli  by  different  subjects  to  be  explicitly  controlled  in  the  processing  of 
the  task  results.  Individuals  performing  any  of  the  above  tasks  will  place 
varying  emphasis  on  the  underlying  distinctions  which  interrelate  stimuli.  The 
algorithms  used  to  process  the  results  of  all  of  the  above  tasks  control  for 
this  type  of  between-subject  variance.  However,  individuals  will  also  vary  in 
the  level  of  detail  they  consider  when  performing  the  task.  For  example,  some 
will  utilize  only  broad  distinctions  and  find  all  the  stimuli  either  very  dif¬ 
ferent  of  very  similar,  while  others  will  utilize  very  few  distinctions  and  find 
each  pair  of  stimuli  similar  in  a  different  degree.  It  is  useful  to  be  able  to 
focus  on  a  specific  level  of  distinction  in  processing  the  results  of  the 
task.  If  only  the  most  general  dimensions  are  sought  from  the  MDS,  then  broad 


distinctions  should  be  emphasized.  If  all  possible  dimensions  are  sought  from 
the  MDS,  than  fine-grained  distinctions  should  be  emphasized.  This  can  be  done 
only  with  unconstrained  sorting  data. 

Second,  the  unconstrained  sort  is  the  simplest  and  fastest  means  of 
obtaining  data  for  MDS.  The  TACCOs  being  interviewed  could  provide  the  needed 
data  for  MDS  in  half  an  hour  if  the  Q-sort  were  used,  but  if  any  other  tech¬ 
niques  were  employed,  several  hours  could  be  required,  just  to  obtain  the  MDS 
data.  Moreover,  because  paired  comparisons,  tracks,  etc.,  require  direct 
interaction  between  the  interviews  and  the  interviewees,  only  one  TACCO  could  be 
interviewed  at  a  time.  The  limited  amount  of  time  available  to  interview  each 
TACCO,  and  the  need  to  interview  several  TACCOs  simultaneously,  favored  the  use 
of  the  unconstrained  sort. 

Extensive  investigation  of  Burton  (1969)  and  M.  Miller  (1975)  has  shown 
that  unconstrained  sorting  produces  equivalent  results  to  other  techniques, 
although  two  general  rules-of -thumb  should  be  obeyed  in  its  application.  The 
first  is  that  the  number  of  stimuli  must  be  greater  than  ten,  and  the  second  is 
that  there  must  be  more  than  twice  as  many  subjects  performing  the  sort  as  there 
are  stimuli  being  sorted.  The  first  condition  was  met,  as  a  total  of  14  deci¬ 
sion  functions  were  used  in  the  analysis  (see  Section  2. 2. 2. 2).  The  second  con¬ 
dition  was  also  met  in  that  a  total  of  32  TACCO's  were  utimately  interviewed. 

2. 2. 2. 2  Stimulus  Set 

In  Zachary  (1980),  the  six  Air  ASW  decision  situations  were  organized 
into  individual  decision  functions  which  arise  in  the  generic  Air  ASW  mission. 
These  decision  functions  were  then  related  to  individual  tasks  which  they 
required  on  the  part  of  the  operator.  The  original  prioritization  of  the  six 
decision  situations  (ibid:  Section  6)  was  based  on  these  individual  decision 
functions.  In  order  to  establish  comparability  between  the  previous  prioritaza- 
tion  and  that  derived  in  this  effort,  it  was  necessary  to  use  these  same  decision 
functions  as  the  stimulus  set  for  the  unconstrained  sorting  task.  An  even  more 
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important  reason  for  using  the  decision  functions  as  the  stimuli  instead  of  the 
decision  situations  was  that  the  decision  situations  are  analytic  constructs. 
They  represent  complex  sets  of  interrelated  decision-making,  information 
processing,  and  communication  functions  on  the  part  of  the  TACCQ.  The  decision 
situations  were  likely  to  have  little  intuitive  meaning  for  the  TACCOs,  without 
considerable  definition  and  explanation.  The  basic  decision  functions,  on  the 
other  hand,  were  directly  relevant  to  the  experience  of  the  TACCOs  because  they 
represent  specific  problems  which  the  TACCOs  can  associate  to  their  on-platform 
operations.  As  a  result,  the  TACCOs  can  apply  their  detailed  knowledge  and 
intuition  about  the  decision  functions,  which  they  would  be  unable  to  do  in  the 
decision  situations. 


Fourteen  decision  functions  were  selected.  Each  is  described  below, 
along  with  a  two-letter  code  used  to  identify  it  on  the  response  forms  for  the 
sorting  task  and  the  subsequent  ranking  task.  The  decision  functions  used  in 
the  sorting  are: 

•  ADJUST  PATTERN  TO  SENSOR  FAILURE  (AP)  -  replacement  of 
faulty  sensors  or  adjustment  of  aircraft  track  due  to 
equipment/sensor  failure. 

•  EXTEND  EXISTING  SENSOR  PATTERN  (EP)  -  determine  orienta¬ 
tion  and  settings  of  sensors  to  be  added  to  existing  pat¬ 
tern  with  no  contact. 

•  ANTICIPATE  TARGET  MOVEMENT  (AM)  -  predict  future  location, 
course,  speed,  and  depth  of  target  based  on  intelligence 
and  actual  tactical  situation. 

•  CONSTRUCT  SENSOR  MONITORING  PATTERN  (MP)  -  determine  where 
to  position  the  aircraft  to  obtain  maximum  reception  from 
sensors,  and  sequence  in  which  to  monitor  deployed  (acoustic) 
sensors  if  number  of  sensors  exceeds  number  of  aircraft 
receivers. 

•  COORDINATE  HAND-OFF  (CH)  -  transmit  data  to  relief  platform 
for  continuation  of  prosecution. 

•  DETERMINE  SIGNAL  IS  A  VALID  CONTACT  (VC)  -  reduce  false 
alarm  rates  of  sensor  system  (e.g.,  clouds  on  radar,  random 
noise  on  MAD  or  acoustic). 
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•  DETERMINE  WEAPON  AND  SETTING  FOR  ATTACK  (DS)  -  select  weapon 
to  be  used  for  the  attack,  and  its  optimum  weapon  settings 
(e.g.,  search  depth,  minimum/maximum  search  limit). 

•  GAIN  ATTACK  CRITERIA  (AC)  -  interpret  sensor  data  to  determine 
when  target  is  localized  sufficently  to  place  an  attack. 

•  DETERMINE  TARGET  FIX  (TF)  -  use  incoming  sensor  data  to 
establish  the  location  of  the  target. 

•  CREATE  SENSOR  PATTERN  (CP)  -  determine  pattern,  spacing, 
orientation  and/or  utilization  for  all  sensor  types  for 
each  phase  of  the  mission. 

•  MANAGE  EQUIPMENT/STORES  TO  ACCOMODATE  PRESENT  AND  FUTURE  NEEDS  (ME) 
monitor  inventory  of  sensors  and  equipment  status  for  applica¬ 
tion  for  future  tactics  and  resources. 

•  CLASSIFY  SIGNAL  (CS)  -  determine  if  signal  (primarily  acoustic) 
may  be  originating  from  the  target  of  interest. 

e  COMPENSATE  FOR  ACOUSTIC/ATMOSPHERIC  PROPAGATION  CONDITIONS  (PC)  - 
determine  the  adjustments  in  sensor  spacing,  aircraft  track, 
aircraft  altitude,  etc.,  that  must  be  made  when  actual  atmospheric 
and  bathythermal  conditions  are  different  from  their  forecast. 

•  DETERMINE  AIRCRAFT  WEAPON  LAUNCH  POSITION  (LP)  -  determine  factors 
to  place  the  aircraft  in  an  optimum  attack  position. 


The  14  decision  functions  are  summarized  in  Table  2-1. 


2. 2. 2. 3  Task  Instructions  and  Materials 

Each  individual  performing  the  sorting  task  was  presented  with  the  stim¬ 
uli  in  the  form  of  a  deck  of  three  by  four  inch  cards.  In  the  center  of  each 
card  (in  bold  type)  was  the  name  of  the  decision.  Immediately  under  the  name 
was  a  short  explanation  of  the  decision  problem,  printed  in  a  lighter  type  face. 
The  two-letter  code  used  to  identify  each  decision  on  the  response-recording 
sheet  was  located  in  the  upper  right  corner  of  the  card.  The  two  letter  codes 
are  mnemonics,  derived  from  the  first  letters  of  keywords  in  the  decision  name. 
The  mnemonic  codes  are  given  in  Table  2-1  next  to  the  name  of  each  decision. 
Mnemonic  codes  were  selected  over  numeric  or  alphabetic  codes  to  identify  the 


Table  2-1.  Fourteen  ASW  Decision  Functions  and  Alphabetic  Codes 


CODE 

DECISION  FUNCTION 

AP 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

EP 

EXTEND  SENSOR  PATTERN 

AM 

ANTICIPATE  TARGET  MOVEMENT 

MP 

CONSTRUCT  SENSOR  MONITORING  PATTERN 

CH 

COORDINATE  HANOOFF 

VC 

DETERMINE  SIGNAL  IS  A  VALID  CONTACT 

OS 

DETERMINE  WEAPON  AND  SETTING  FOR  ATTACK 

AC 

GAIN  ATTACK  CRITERIA 

TF 

DETERMINE  TARGET  FIX 

CP 

CREATE  SENSOR  PATTERN 

ME 

MANAGE  EQUIPMENT/STORES  TO  ACCOMODATE  PRESENT  AND  FUTURE  NEEDS 

CS 

CLASSIFY  SIGNAL 

PC 

COMPENSATE  FOR  IN  SITU  ACOUSTICAL 

AND  ATMOSPHERIC  PROPAGATION  CONDITIONS 

LP 

DETERMINE  AIRCRAFT  LAUNCH  POSITION  FOR  ATTACK  ON  TARGET 
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decisions  because  number  or  letter  codes  could  affect  the  way  of  sorting  was 
conducted  by  suggesting  an  implicit  order  which  was  not  intended. 


The  sorting  task  was  performed  as  follows.  After  the  initial  briefing, 
each  TACCO  was  given  a  pack  of  cards  and  a  set  of  response  forms  (reproduced  in 
Appendix  B).  The  first  response  sheet  requested  background  data  on  the 
experience  of  the  respondent;  the  interviewees  were  asked  to  fill  in  these  data 
before  proceeding.  They  were  then  required  to  read  the  instructions  on  their 
response  forms  (see  Appendix  B)  and  perform  the  sorting  task.  There  were  no 
time  limitations;  no  subject  required  more  than  30  minutes. 


2. 2. 2. 4  The  Ranking  Task 

The  second  of  the  two  psychometric  tasks  performed  by  the  TACCOs  was  to 
rank  order  the  stimuli  used  in  the  unconstrained  sorting.  The  subjects  were 
presented  with  a  criterion  and  asked  to  rank  order  the  items  according  to  that 
criterion.  This  task  was  repeated  four  times,  using  four  different  criteria: 


•  Importance  of  the  decision  in  a  mission  with  an  attack 
on  the  submarine, 

•  Importance  of  the  decision  in  a  mission  with  only  sur¬ 
veillance  of  the  submarine, 

•  Urgency  of  the  decision,  and 

•  TACCO  workload  during  the  decision. 


The  first  two  rankings  provided  the  primary  inputs  for  the  unfolding 
analysis.  These  two  rankings  used  the  same  criterion  —  the  perceived  impor¬ 
tance  of  the  decision  functions  to  the  mission.  The  difference  between  the  two 
rankings  was  the  type  of  mission  used  as  a  reference.  In  the  first  ranking  the 
objective  of  the  mission  was  to  locate,  attack,  and  destroy  the  hostile 
submarine.  In  the  second  ranking  the  objective  of  the  mission  was  to  locate, 
track,  and  handoff  the  hostile  submarine  to  a  relief  platform.  In  both  of  these 
rankings,  the  criterion  of  importance  was  defined  as  the  degree  to  which  a  less- 
than-optimal  decision  would  negatively  impact  on  the  achievement  of  the  mission 


objective. 


The  third  ranking  was  performed  using  a  criterion  of  urgency.  Urgency 
was  defined  as  the  relative  speed  with  which  the  decision  had  to  be  made,  once 
it  became  apparent  that  the  decision  was  required.  The  fourth  ranking  used  a 
criterion  of  workload.  The  term  workload  was  only  minimally  defined.  The  sub¬ 
jects  were  instructed  to  consider  mental  (cognitive)  workload  as  well  as  manual 
(physical)  workload.  These  last  two  rankings  were  included  to  allow  future  MDS 
analysis  of  perceived  decision  aiding  needs  against  perceived  time  and  operator 
workload  constraints  during  the  air  ASW  mission,  and  to  provide  supplemental 
information  which  could  aid  in  the  interpretation  of  the  MDS  solution  derived 
here. 

In  performing  the  ranking  task,  the  subjects  used  the  same  cards  con¬ 
taining  the  decision  function  descriptions  that  they  used  in  performing  the 
unconstrained  sorting  task.  Their  instructions  simply  identified  the  criteria 
on  which  they  were  to  rank  the  stimuli.  The  instructions  and  response  forms  are 
contained  in  Appendix  B. 

2.2.3  Final  Discussions 

Following  the  final  ranking  tasks,  the  response  forms  were  collected 
and  comments  were  solicited  from  the  TACCOs  with  regard  to: 

•  decision  aids  in  general,  and  the  Naval  Air  ASW  decision 
aid  effort  in  particular, 

•  the  tasks  which  they  had  just  performed  and  the  decision 
functions  used  in  the  tasks,  and 

•  general  conments/suggestions  concerning  decision  making  and 
human  factors  engineering  problems  on-board  the  P-3C  aircraft 
which  they  have  flown. 

Most  of  the  discussions  focused  on  the  last  topic  lasted  above.  The  points  made 
in  these  the  discussions  are  summarized  in  Appendix  E. 
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INTERVIEW  RESULTS  AND  PRELIMINARY  ANALYSIS 

Prior  to  the  interviews  at  Moffett  Field,  a  series  of  preliminary  inter¬ 
views  were  conducted  at  the  Naval  Air  Development  Center  (NADC).  The  interviewees 
were  NADC  personnel  with  subtantial  experience  as  TACCO's  on  the.  P-3C.  One  indi¬ 
vidual  with  S-3A  experience  was  also  interviewed.  These  interviews  were  con¬ 
ducted  to  identify  and  correct  in  advance  any  problems  in  the  multidimensional 
scaling  interview  structure.  The  inteviewees  were  asked  to  identify  any  errors 
or  inadequacies  in  either  the  interview  structure  or  the  interview  material.  As 
problems  were  identified,  they  were  corrected  before  the  next  set  of  preliminary 
interviews.  This  procedure  produced  the  following  change  in  the  interview  struc¬ 
ture  and  materials: 

•  Several  decision  functions  were  added  and/or  deleted  from 
the  stimulus  set,  e.g.  Damage  Assesment  was  deleted  and 
Coordinate  Handoff  was  added, 

•  More  details  were  added  to  the  initial  briefing,  e.g.  more 
examples  of  decision  aids  were  included, 

•  A  single  ranking  by  importance  was  replaced  by  dual  rankings 
considering  ASW  missions  with  and  without  attack  on  the 
submarine, 

•  Importance  and  urgency  were  included  as  separate  ranking 
criteria,  and 

•  The  final  definition  of  importance  was  constructed. 

All  of  these  changes  are  reflected  in  the  description  of  the  interview  procedure 
given  in  Section  2.2  above. 

2.3.1  Preprocessing  of  Sorting  Data  for  MDS  Program 

Interviews  were  conducted  with  a  total  of  32  TACCOs  at  NAS,  Moffett 
Field  over  a  five  day  period  (10  December  1979  to  14  December  1979).  A  separate 
interiew  session  was  conducted  each  morning  and  afternoon.  Sorting  data  were 
obtained  from  31  of  the  32  interviewees;  one  person  misinterpreted  the  instruc¬ 
tions  and  produced  unusable  data.  These  31  sortings  are  listed  in  Appendix  C. 


The  sortings  were  prepared  for  input  to  the  MDS  computer  program  by  an  algorithm 
developed  by  Burton  (1975),  which  processes  unconstrained  sorting  data  to  pro¬ 
duce  a  dissimilarity  measure  among  the  decisions  in  the  stimulus  set.  In 
general,  a  dissimilarity  measure  is  a  binary  mapping  from  the  set  of  objects 
scaled  to  some  numerical  scale.  For  any  two  objects  i  and  j,  the  dissimilarity 
between  i  and  j  is  written  as  d-jj.  Dissimilarity  measures  are  analogous  to 
topological  metrics,  and  have  three  main  properties: 

(1)  djj  *  0  iff:  i=j,  d-j j  >0  otherwise  (positivity) 

(2)  dij  =  dj-j  for  all  i,j  (symmetry) 

(3)  dij  +  dj|<,  for  all  i,j,k  (triangle  inequality) 

Dissimilarity  measures  are  normally  represented  as  a  square  matrix  of  values 
(with  each  row/column  representing  one  of  the  stimuli  from  the  sorting)  so  the 
terms  dissimilarity  '‘measure"  and  dissimilarity  "matrix"  are  used  here 
interchangeably. 

Burton's  algorithm  computes  the  dissimilarity  between  two  stimuli 
according  to  the  number  of  individuals  who  placed  them  in  different  groups,  and 
the  values  that  are  assigned  to  two  parameters,  aand  e.  The  first  parameter,  a, 
represents  the  level  of  discrimination  selected  for  the  analysis  (see  Section 
2. 2.2.1  above).  The  second  parameter,  e,  represents  the  minimal  dissimilarity 
between  two  stimuli  (i.e.,  that  dissimilarity  which  results  when  two  simuli  are 
placed  in  the  ocam  group  by  one  individual).  The  algorithm  is  based  on  the 
mathematical  principle  that  each  individual  induces  a  partition  on  the  set  of 
stimuli  by  placing  them  into  mutually  exclusive  groups.  Rj  is  defined  as  the 
number  of  groups  formed  by  individual  i,  (i.e.  the  number  of  cells  in  the  par¬ 
tition  induced  by  subject  i),  and  is  defined  as  the  number  of  stimuli  placed 


in  group  k  by  subject  i.  If  the  number  of  subjects  is  T,  then  dissimilarity 
between  stimuli  x  and  y  is: 

T 

dXy  a  1*  (C-Si(x,y)), 

where  S-j  (x,y)=Aj|<  if  subject  i  placed  stimuli  x  and  y  in  cell  k  of  the  partition, 

=Bi  if  subject  i  placed  stimuli  x  and  y  in  different  cells  of  the 
partition 

=C  if  x=y. 

There  are  two  constraints  on  d: 

1)  Bi  <  min(Ai|c),  over  all  k 

2)  C  >  maxfAik),  0ver  an 

A  useful  measure  is  obtained  by  defining 

Aik=  (Mik)° 

Bi  *  0 

C  =  1+  e,  if  a<  =0 
=  N+  e,  if  ct*  0 
*  2+  e,  if  <*<  0 

Since  e  is  a  minimum  value  used  to  differentiate  items  placed  together  by  a  given 
individual,  it  should  be  very  small,  a  represents  the  level  of  distinction  used 
by  the  individuals  doing  the  sortings.  If  a  is  greater  than  zero,  a  greater 
weight  is  given  to  sortings  using  a  few,  large  cells,  and  the  resulting  distance 
measure  tends  to  be  the  lower  dimensionality  when  subjected  to  multidimensional 
scaling.  If  a  is  less  than  0,  then  a  greater  weight  is  given  to  partitions  which 
include  a  greater  number  of  small  cells,  and  the  measure  results  in  higher¬ 
dimensional  multidimensional  scaling  solutions.  When  a  is  0,  there  is  equal 
weight  given  to  large  and  small  cell  sizes,  as  1  is  always  added  to  dXy  whenever 
x  and  y  are  placed  in  different  cells,  and  e  is  always  added  to  dXy  when  they  are 
placed  in  the  same  cell.  nr 
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A  computer  program  named  METRIC  which  performs  this  algorithm  was  used 
to  produce  a  dissimilarity  measure  from  the  results  of  unconstrained  sortings. 
Because  the  intent  is  to  identify  only  the  most  salient,  important  dimensions  in 
ASW  decision-making,  an  a  value  of  -.5  was  used.  The  resulting  dissimilarity 
matrix  is  shown  in  Table  2-2.  The  decision  functions  are  identified  in  Table  2-2 
by  the  two-letter  codes  depicted  earilier  in  Table  2-1.  Only  the  lower  triangle 
of  the  matrix  is  shown  because  the  matrix  is  symmetrical  and  the  diagonal  con¬ 
tains  only  zeroes. 

2.3.2  Preliminary  Analysis  of  Ranking  Results 

Rankings  of  the  14  decision  functions  by  the  four  criteria  were  obtained 
from  all  32  TACCOs  interviewed  at  Moffett  Field.  These  rankings  are  given  in 
Appendix  0.  The  sorting  task  data  required  preprocessing  via  Burton's  algorithm 
before  they  could  be  used  in  the  MDS  analysis,  but  the  ranking  task  data  could  be 
input  directly,  to  the  unfolding  analysis  procedure.  It  was  possible  however,  to 
perform  several  useful  preliminary  analyses  of  the  rankings  prior  to  their  use  in 
the  unfolding  analysis. 

The  first  preliminary  analysis  allowed  the  statistical  significance  of 
the  ranking  data  to  be  assessed.  Kendall's  W,  the  coefficient  of  concordance,  is 
a  statistic  which  measures  the  degree  of  agreement  among  K  rankings  of  the  same 
set  of  N  objects.  A  test  of  significance  of  the  concordance  among  the  rankers 
(here,  the  TACCOs)  can  be  based  on  the  value  of  W  (Kendall,  1962).  This  test 
uses  the  null  hypothesis  that  there  is  no  agreement  among  the  different  rankings, 
and  is  based  on  the  fact  that  K(N-1)W  is  asymptotically  distributed  according  to 
the  Chi-Square  distribution  with  N-l  degrees  of  freedom  (Gibbons,  1971). 

The  coefficient  of  concordance  was  computed  and  its  significance  tested 
for  each  of  the  four  rankings  obtained  from  the  TACCOs  at  Moffett  Field.  The 
results  are  shown  in  Table  2-3.  All  four  rankings  were  found  to  be  significant 


at  the  .01  level,  allowing  the  null  hypothesis  to  be  rejected  with  greater  than 
99%  confidence.  These  tests  indicated  that 


•  The  TACCOs  were  not  performing  the  ranking  tasks  in  a 
random  or  careless  fashion,  and 

•  The  TACCOs  were  ranking  the  tasks  in  a  consistent  way 
for  each  criterion. 


The  rankings  were  thus  shown  to  contain  significant  data  for  the  unfolding 
analysis. 


Table  2-3.  Significance  of  Inter-Subject  Agreement  in  Pour  Ranks  of  ASW  Decisions 


RANKING 

CRITERION 

KENDALL'S  W 
(COEFFICIENT  OF 
CONCORDANCE) 

X2  (CHI-SQUARE) 
BASED  ON  W 

REJECTION  LEVEL 

FOR  NULL  HYPOTHESIS 

OF  NO  AGREEMENT 
AMONG  RANKINGS 

IMPORTANCE  IN 
MISSION  WITH 
ATTACK 

0.39496 

164.305 

<001 

IMPORTANCE  IN 
MISSION  WITHOUT 
ATTACK 

0.49067 

204.118 

<.001 

URGENCY 

0.50768 

211.196 

<.001 

WORKLOAD 

0.28419 

118.221 

<001 
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The  second  preliminary  analysis  conducted  on  the  ranking  results  pro¬ 
vided  an  overall  pattern  of  responses  for  each  criterion.  Average  rankings  were 
constructed  for  each  criterion  by  simply  averaging  the  ranks  assigned  to  each 
decision  function  by  each  of  the  32  TACCOs.  These  average  rankings  are  shown  in 
Tables  2-4  through  2-7.  As  these  tables  show,  each  criterion  produced  a 
substantially  different  ranking,  confirming  the  assumption  that  multiple  cri¬ 
teria  were  needed  to  map  the  Implicit  prioritization  of  the  decision  functions 
by  the  TACCOs. 


Table  2-4.  Average  Ranking  of  Decisions  in  Mission  With  Attack 


DECISION  NAME 

AVERAGE 

RANK 

CLASSIFY  SIGNAL 

3.63 

(INCREASING) 

GAIN  ATTACK  CRITERIA 

4.19 

DETERMINE  TARGET  FIX 

4.94 

DETERMINE  AIRCRAFT  WEAPON 

LAUNCH  POSITION 

5.78 

DETERMINE  SIGNAL  IS  VALID  CONTACT 

6.25 

CREATE  SENSOR  PATTERN 

6.31 

ANTICIPATE  TARGET  MOVEMENT 

6.44 

IMPORTANCE 

DETERMINE  WEAPON  AND  SETTING 

FOR  ATTACK 

6.84 

COMPENSATE  FOR  IN-SITU  PROPAGATION 

CONDITIONS 

8.34 

MANAGE  EQUIPMENT/STORES 

9.44 

CONSTRUCT  SENSOR  MONITORING  PATTERN 

9.75 

EXTEND  SENSOR  PATTERN 

9.97 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

1050 

1 

r 

COORDINATE  HAND-OFF 

12.63 

(DECREASING) 
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Table  2-5.  Average  Ranking  of  Decisions  in  Mission  Without  Attack 


DECISION  NAME 

AVERAGE 

RANK 

CLASSIFY  SIGNAL 

3.25 

(INCREASING) 

CREATE  SENSOR  PATTERN 

4.76 

ANTICIPATE  TARGET  MOVEMENTS 

4.97 

DETERMINE  TARGET  FIX 

5.22 

COORDINATE  HAND-OFF 

5.34 

DETERMINE  SIGNAL  IS  A  VALID  CONTACT 

5.41 

COMPENSATE  FOR  IN-SITU  PROPAGATION 

CONDITION 

7.09 

IMPORTANCE 

MANAGE  EQUIPMENT/STORES 

7.44 

EXTEND  SENSOR  PATTERN 

8.13 

CONSTRUCT  SENSOR  MONITORING 

PATTERN 

8.16 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

8.62 

GAIN  ATTACK  CRITERIA 

10.81 

DETERMINE  AIRCRAFT  WEAPON  LAUNCH 

POSITION 

12.88 

DETERMINE  WEAPON  AND  SETTING  FOR 

1 

< 

ATTACK 

12.94 

(DECREASING) 

Table  2-6.  Average  Ranking  of  Decisions  By  Urgency 


DECISION  NAME 

AVERAGE 

RANK 

DETERMINE  AIRCRAFT  WEAPON  LAUNCH 

’ 

POSITION 

2.59 

(INCREASING) 

GAIN  ATTACK  CRITERIA 

3.19 

i 

DETERMINE  WEAPON  AND  SETTING  FOR 

ATTACK 

4.56 

ANTICIPATE  TARGET  MOVEMENT 

5.25 

DETERMINE  TARGET  FIX 

5.66 

DETERMINE  SIGNAL  IS  VALID  CONTACT 

6.63 

URGE 

MCY 

CLASSIFY  SIGNAL 

7.09 

MANAGE  EQUIPMENT/STORES 

7.94 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

8.56 

COMPENSATE  FOR  IN-SITU  PROPAGATION 

CONDITIONS 

9.97 

CREATE  SENSOR  PATTERN 

10.22 

EXTEND  SENSOR  PATTERNS 

10.38 

COORDINATE  HAND-OFF 

11.09 

CONSTRUCT  SENSOR  MONITORING 

▼ 

PATTERN 

11.88 

(DECREASING) 

[i] 


Table  2-7.  Average  Ranking  of  Decisions  By  Workload 


DECISION  NAME 

AVERAGE 

RANK 

GAIN  ATTACK  CRITERIA 

3.94 

(HIGHER) 

DETERMINE  AIRCRAFT  WEAPON  LAUNCH 

i 

i 

POSITION 

4.66 

DETERMINE  TARGET  FIX 

5.13 

ANTICIPATE  TARGET  MOVEMENT 

5.94 

COORDINATE  HAND-OFF 

6.00 

DETERMINE  WEAPON  AND  SETTING  FOR 

ATTACK 

6.44 

WORKLOAD 

MANAGE  EQUIPMENT/STORES 

6.56 

CREATE  SENSOR  PATTERNS 

7.94 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

9.31 

COMPENSATE  FOR  IN-S1TU  PROPAGATION 

CONDITIONS 

9.34 

EXTEND  SENSOR  PATTERNS 

9.38 

DETERMINE  SIGNAL  IS  VALID  CONTACT 

9.66 

CLASSIFY  SIGNAL 

10.09 

CONSTRUCT  SENSOR  MONITORING 

i 

PATTERN 

10.63 

(LOWER) 

I 


(Xi 


3.  DIMENSIONAL I ZING  THE  ASW  DECISION  SPACE 


The  unconstrained  sorting  task  provides  data  on  the  differences  per¬ 
ceived  by  TACCOs  among  a  sample  of  fourteen  ASW  decision  functions.  The 
judgments  on  the  similarities  and  differences  of  the  decisions  reflect  the 
underlying  distinctive  features  or  dimensions  by  which  the  TACCO  organize  their 
perceptions  of  ASW  decision  making  situations.  The  interrelationships  of  the  14 
decisions  in  the  stimulus  set  on  these  underlying  dimensions  are  reconstructed 
from  the  unconstrained  sorting  data  through  Multidimensional  Scaling  techniques. 
These  mathematically  reconstructed  relationships  identify  the  basic  principles 
which  the  experienced  TACCOs  used  when  they  performed  the  sortings,  and  thus 
dimensionalize  the  TACCOs'  perceptual  decision  space. 

This  section  details  the  MDS  procedures  used  to  dimensionalize  the  ASW 
decision  space  and  presents  the  results  of  that  dimensionalization.  Section  3.1 
briefly  reviews  the  principals  of  metric  and  non-metric  MDS.  Section  3.2  pre¬ 
sents  the  results  of  the  MDS  analysis  of  the  Moffett  Field  data  and  Section  3.3 
provides  some  interpretations  of  the  three  dimensions  that  were  obtained. 

3.1  NON-METRIC  MULTIDIMENSIONAL  SCALING  TECHNIQUES 

Multidimensional  Scaling  techniques  are  mathematical  procedures  that 
reconstruct  a  topological  space  (i.e.  a  set  of  real-valued  orthogonal  dimensions 
and  a  distance  metric)  which  best  models  a  set  of  observed  inter-object  distance 
measurements  for  some  group  of  objects.  If,  for  example,  three  objects  are 
observed  to  be  one  unit  apart  from  one  another,  then  their  interrelationships 
can  be  optimally  represented  as  the  vertices  of  an  equilateral  triangle.  Since 
a  triangle  is  two-dimensional,  the  underlying  topological  space  is 
two-dimensional.  A  one-dimensional  space  could  not  be  used  because  it  is 
impossible  to  place  three  points  on  a  line  such  that  they  are  one  measurement 


unit  apart.  A  three  or  higher  dimensional  solution  is  possible  but  not  optimal 


because  the  distances  in  the  example  can  be  modeled  completely  in  two  dimensions 


so  additional  dimensions  would  be  extraneous.  It  is  therefore  possible  to 


represent  the  three  objects  as  points  in  a  two-dimensional  space  such  that  two 


conditions  are  met: 


1.  the  distances  between  the  pairs  of  the  objects  are  all  equal  to 
the  values  originally  observed,  and 

2.  the  space  is  unique  up  to  a  translation  or  rotation  of  axes. 


These  conditions  are  the  goals  of  an  MDS  analysis.  They  are  thus  also  the  cri¬ 
teria  by  which  an  MDS  result  is  evaluated.  In  this  simple  example,  the  actual 
numerical  values  assigned  to  the  original  (observed)  distances  are  crucial  to 
determining  and  verifying  the  solution,  so  this  approach  is  known  as  metric  MDS. 


Metric  MDS  is  similar  to  factor  analysis  and  has  been  in  use  for  over 
50  years.  Its  use  requires  that  the  numerical  values  which  represent  the 
observed  distances  have  the  properties  of  a  true  ratio  scale.  This  condition  is 
seldom  met  in  psychological  and  decision  science  research;  often  the  best 
available  data  have  only  the  properties  of  ordinal  scales.  Therefore,  in  the 
mid  1960's  several  researchers  independently  developed  a  non-metric  version  of 
Multidimensional  Scaling.  Non-metric  MDS,  unlike  metric  MDS,  uses  only  the 
ordinal  properties  of  the  observed  distances  between  the  objects  being  scaled. 
This  allows  a  more  general  interpretation  of  the  distance  values.  In  non-metric 
MDS,  any  measure  of  di8eimilarity  can  be  used  as  an  estimate  of  the  true 
interobject  distances  as  long  as  it  is  monotonically  related  to  a  true  distance 
measure. 


Non-metric  MDS  permits  reconstruction  of  a  real-valued  multidimensional 
structure  from  ordinal  data  through  the  use  of  ordinal  constraints.  In  general, 
when  only  ordinal  properties  of  data  are  used  in  a  geometrical  procedure,  it  is 
possible  to  define  a  solution  only  in  terms  of  possible  regions  of  the  solution 
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space.  But  as  additional  constraints  are  placed  on  the  problem  (even  ordinal 
constraints)  it  becomes  possible  to  further  restrict  the  region  of  the  solution 
space  where  the  true  solution  exists.  If  a  great  many  constraints  are  involved, 
it  may  be  possible  in  practice  to  restrict  the  solution  region  so  much  that  it 
can  be  treated  as  a  specific  point  in  the  solution  space,  i.e.  as  an  absolute 
solution.  This  is  how  non-metric  MDS  works. 

The  constraints  arise  from  the  nature  of  the  dissimilarity  measure.  If 
there  are  n  objects  being  scaled,  then  there  will  be  (I))  unique  dissimilarity 
measurements  among  them.  However  the  direction  of  relationship  (i.e.  less 
than,  greater  than,  or  equal)  between  each  pair  of  these  measurements  must  be 
maintained  in  any  MDS  solution.  These  pairwise  measurements  comparisons  are  (by 
definition)  the  ordinal  structure  of  the  data.  Each  comparison  thus  represents 
an  ordinal  constraint  on  the  MDS  procedure,,  since  it  is  an  ordinal  relationship 
which  must  be  preserved  in  the  solution.  Because  there  are  (£)  unique  measure¬ 
ments,  there  are^( 2)^  unique  ordinal  constraints,  a  very  large  number  when 
compared  with  the  number  of  objects  being  scaled  (n).  In  an  MDS  problem,  the 
number  of  constraints  is  usually  so  much  larger  than  n  that  a  unique  MDS  solu¬ 
tion  can  be  computed  without  using  the  actual  numerical  values  of  the 
dissimilarities. 


MDS  algorithms  operate  in  a  three-stage  iterative  procedure.  In  the 
first  stage,  the  dissimilarity  matrix  for  the  stimuli  (e.g.  Table  2-2)  is  used 
to  construct  an  initial  representation  of  the  stimuli  as  points  in  a  multidimen¬ 
sional  space.  This  is  termed  the  initial  configuration.  The  dissimilarity 
measure  is  also  used  to  construct  the  set  of  ordinal  constraints  which  will 
govern  the  remainder  of  the  procedure. 


In  the  second  stage,  the  algorithm  measures  the  goodness  of  the  fit 
between  the  present  configuration  of  the  stimuli  in  the  space  and  the  ordinal 
constraints  through  some  statistical  or  heuristic  criterion.  A  heuristic 
criterion  is  one  that  has  proven  useful  in  identifying  improvement  in  one 
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solution  over  another  but  which  has  no  statistical  basis  and  which  has  no  known 
sampling  distribution.  Thus,  a  change  in  a  heuristic  criterion  can  be  easily 
interpreted  as  indicating  an  improvement  (or  deterioration,  depending  on  the 
direction  of  the  change)  in  the  solution  but  can  not  provide  an  indication  of 
the  significance  of  the  change.  A  statistical  criterion  is  one  that  provides  a 
statistical  measure  of  improvement.  The  most  common  statistical  criterion  is 
the  coefficient  of  variation  (r^)  which  measures  the  proportion  of  the  variance 
in  the  data  that  is  accounted  for  in  the  solution.  Since  non-metric  MDS 
algorithms  do  not  use  the  actual  dissimilarity  values  in  testing  the  goodness  of 
fit,  statistical  criteria  such  as  r^  are  usually  impossible  to  compute. 


In  the  third  stage  of  an  MDS  algorithm  the  location  of  each  stimulus  on 
each  dimension  is  adjusted  so  as  to  increase  the  goodness  of  the  fit  on  the  next 
iteration.  The  second  and  third  stages  are  iterated  until  either  the  criterion 
reaches  some  threshold  value  or  a  predefined  number  of  iterations  has  been 
completed. 


The  most  commonly  used  criterion  in  non-metric  MDS  is  a  heuristic  cri¬ 
terion  termed  "stress."  Stress  is  a  measure  of  the  relative  number  of  ordinal 
constraints  that  are  violated  by  the  current  solution.  The  primary  MDS 
algorithm  used  in  the  present  analysis  employed  the  following  stress  criterion: 


STRESS  = 


( 1  >3 ) 


where  d^j  is  the  distance  between  stimuli  i  and  j  in  the  multidimensional 
scaling  solution,  d*j  is  the  dissimilarity  between  stimuli  i  and  j  used  as  the 
input  to  the  MDS  procedure,  and  f(d,j)  is  a  monotonic  tranformation  of  d^j,  sub¬ 
ject  to  the  general  constraint: 

ftd^)  >  f(dkl)  whenever  >  dkl> 

fK) 
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The  function  f(d„)  can  be  obtained  in  a  variety  of  ways,  but  in  the  algorithm 
used  in  this  study,  it  was  calculated  through  Kruskal's  monotone  regression  pro¬ 
cedure  (see  Roskam  1977). 

Some  MDS  algorithms  use  a  metric  procedure  similar  to  factor  analysis 
to  derive  the  initial  configuration  but  then  proceed  nonmetrically  through  the 
remainder  of  the  algorithm.  These  algorithms  are  termed  "quasi -metric''  MDS 
algorithms.  True  non-metric  algorithms  obtain  the  initial  configuration  either 
through  a  random  procedure  or  directly  from  the  user. 

The  configuration  of  the  stimuli  in  the  multidimensional  space  when  the 
algorithm  terminates  constitutes  the  MDS  solution.  This  final  configuration 
represents  the  placement  of  the  stimuli  in  the  space  which  best  fits  the 
constraints  on  their  interrelations  imposed  by  the  original  dissimilarity 
measure.  The  criterion  for  algorithm  termination  is  based  only  upon  the  com¬ 
parison  of  the  distances  between  the  stimuli  in  the  MDS  solution  and  the  origi¬ 
nal  dissimilarities.  These  comparisons  are  independent  of  the  placement  of  the 
axes  in  the  multidimensional  space.  It  is  therefore  sometimes  appropriate  to 
rotate  the  axes  of  the  MDS  solution  to  obtain  a  placement  of  the  stimuli  with 
regard  to  the  axes  which  permits  easier  interpretation. 

An  MDS  algorithm  can  only  compute  a  solution  for  a  space  with  a  prede¬ 
fined  dimensionality.  It  is  thus  necessary  to  perform  the  analysis  for  spaces 
of  varying  number  of  dimensions  and  afterward  select  the  solution  which  is  best. 
This  selection  is  normally  made  on  the  basis  of  a  plot  of  the  criterion  (e.g. 
stress)  against  the  dimensionality  of  the  space.  The  stress  associated  with  an 
MDS  solution  will  normally  be  lower  when  more  dimensions  are  used  but  the  rate 
of  decrease  of  stress  with  regard  to  dimensionality  will  vary.  There  will 
usually  be  some  point  (i.e.  number  of  dimensions)  where  the  curve  created  by  the 
connecting  points  in  the  plot  inflects  or  "elbows."  The  dimensionality  at  this 
point  is  taken  as  representing  the  best  solution.  When  the  stress/dimensionality 


plot  shows  a  constant  (linear)  decrease,  it  indicates  that  the  data  being  scaled 
do  not  have  an  underlying  multidimensional  structure  and  MDS  analysis  is 
inappropriate. 

3.2  APPLICATION  Of  MDS  ALGORITHM  TO  THE  MOFFETT  FIELD  TACCO  DATA 

The  dissimilarity  matrix  for  the  fourteen  ASW  decision  functions  (Table 
2-2)  constructed  from  the  unconstrained  sortings  of  the  TACCOs  interviewed  at 
Moffett  Field  was  used  as  input  to  a  Multidimensional  Scaling  program.  The  MDS 
analysis  was  conducted  on  the  University  of  Pennsylvania  Wharton  School's  DEC-10 
computer  using  the  MDSX  package  of  Multidimensional  Scaling  programs  (Coxon  et 
al  1977).  This  package  contains  several  different  MDS  algorithms.  Specifically, 
it  includes  the  MINISSA  algorithm  of  Lingoes  and  Roskam  (1975),  the  INDSCAL 
algorithm  of  Carroll  and  Chang  (1970),  and  Young's  (1968)  TORSCA  IV  algorithm. 
Initially,  each  of  those  algorithms  were  used  on  the  Moffett  Field  data  to  test 
for  convergent  val idit/  among  the  various  programs.  Virtually  identical  results 
were  obtained  from  all  three’  algorithms.  In  the  prioritization  analysis  in 
Section  4.2,  the  output  from  MINISSA  is  used  as  the  primary  MDS  solution  (i.e. 
in  place  of  the  solutions  provided  by  TORSCA  IV  and  INDSCAL)  because  TORSCA  IV 
and  INOSCAL  algorithms  are  both  quasi-metric,  whereas  MINISSA  is  totally 
non-metric. 

Each  MDS  algorithm  was  used  to  construct  successive  solutions  from  one 
to  five  dimensions.  The  plot  of  the  stress  against  the  dimensional ity  for 
MINISSA,  shown  in  Figure  3-1,  indicates  that  a  three  dimensional  solution  is 
best.  Roskam  (1977)  provides  some  "rules-of-thumb"  for  assessing  the  goodness 
of  a  MINISSA  solution  according  to  the  stress  of  the  final  configuration.  He 
indicates  that  a  stress  between  .05  and  .01  should  be  considered  as  "good," 
while  stress  less  than  .01  should  be  considered  the  very  best  quality,  in  other 
words  “execellent."  Thus,  with  a  stress  of  .002,  the  three  dimensional  MINISSA 
solution  can  be  evaluated  as  excellent. 


STRESS 


Figure  3-1.  Minissa  Stress  Vs.  Dimensionality  Plot 
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The  parallel  analysis  of  this  data  with  the  INDSCAL  algorithm  allowed 
a  statistical  estimate  of  the  goodness  of  the  three  dimensional  solution  as 
INDSCAL  uses  the  statistical  criterion  of  r*2  (the  coefficient  of  variation)  for 
evaluating  a  configuration.  The  three  dimensional  INDSCAL  solution  was  found  to 
account  for  76.2%  of  the  variance  in  the  original  dissimilarity  data.  This 
figure  was  in  keeping  with  the  evaluation  of  the  MINISSA  solution. 


The  three  dimensional  MINISSA  solution  is  plotted  in  Figure  3-2.  This 
figure  shows  two  dimensional  plots  of  each  possible  pair  of  the  three  MINISSA 
dimensions.  The  locations  of  the  decision  functions  within  this  multidimen¬ 
sional  space  are  shown  in  Figure  3-2  only  relative  to  each  other  —  no  numerical 
values  are  provided  for  any  of  the  dimensional  scales.  The  actual  coordinates 
of  the  fourteen  decision  functions  on  these  three  dimensions  are  snown  in  Table 
3-1. 

Table  3-1.  Coordinates  of  Decision  Functions  in  Three-Dimensional  Solution 


DECISION  NAME 

LOCATION  ON 
DIMENSION  1 

LOCATION  ON 
DIMENSION  2 

LOCATION  ON 

DIMENSION  3 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

-.14 

-.08 

27 

EXTEND  SENSOR  PATTERN 

-.32 

.02 

-.02 

ANTICIPATE  TARGET  MOVEMENT 

-.16 

.33 

-.41 

CONSTRUCT  SENSOR  MONITORING 
PATTERN 

-.27 

-.02 

.06 

COORDINATE  HAND-OFF 

.36 

.05 

.33 

DETERMINE  SIGNAL  IS  VALID  CONTACT 

.47 

-.55 

-26 

DETERMINE  WEAPON  AND  SETTING  FOR 
ATTACK 

.21 

.17 

.29 

GAIN  ATTACK  CRITERIA 

-.10 

.35 

-.21 

DETERMINE  TARGET  FIX 

-.13 

.26 

-.47 

CREATE  SENSOR  PATTERN 

-.27 

-.02 

.07 

MANAGE  EQUIPMENT/STORES 

-.03 

-.11 

.35 

CLASSIFY  SIGNAL 

.47 

-.55 

-26 

COMPENSATE  FOR  IN-SITU  PROPAGATION 
CONDITIONS 

-.23 

-.05 

.10 

DETERMINE  AIRCRAFT  WEAPON  LAUNCH 
POSITION 

■ 

15 

.21 

.14 

©© 


Figure  3-2.  Plots  of  Decision  Functions  on  Three  Dimensions 


3.3  DIMENSIONAL  INTERPRETATIONS 

The  MDS  algorithm  reconstructs  the  key  underlying  features  or  charac¬ 
teristics  of  the  decision  functions  in  the  stimulus  set  which  the  TACCO's  used 
in  performing  the  unconstrained  sorting  task.  The  algorithm  represents  these 
underlying  features  as  the  dimensional  axes  in  the  MDS  solution.  The  algorithm 
cannot  identify  the  meaning  of  these  dimensional  axes,  however.  The  meanings 
must  be  interpreted  through  a  careful  examination  of  the  way  in  which  the  deci¬ 
sion  functions  in  the  stimulus  set  project  onto  each  axis  of  the  MDS  solution. 
These  projections  are  the  dimensional  coordinates  given  in  Table  3-1.  In  the 
following  sections  each  of  the  dimensions  Is  analyzed  and  a  tenative  interpreta¬ 
tion  offered. 

It  should  be  noted  that  the  multidimensional  space  constructed  by  an 
MDS  algorithm  is  unique  only  up  to  a  translation  or  rotation  of  the  axes.  Thus, 
the  three  dimensions  in  the  MINISSA  solution  for  the  Moffett  Field  unconstrained 
sorting  data  can  be  rotated  to  any  orientation  in  that  space  in  order  to  obtain 
a  more  interpretable  set  of  projections.  The  interpretations  given  below  are 
based  on  the  "unrotated"  orientation  given  in  Table  3-1  and  Figure  3-2  for  two 
reasons.  First,  that  solution  actually  represents  an  implicit  rotation  performed 
by  the  MDS  program  to  minimize  the  correlation  between  dimensional  axes.  This 
provides  the  orientation  of  the  dimensions  in  which  the  axes  are  most 
independent  in  terms  of  the  decision  functions  in  the  stimulus  set.  This  in 
turn,  enhances  the  interpretation  of  the  dimensions  as  independent  features. 
Second,  a  parallel  analysis  was  conducted  in  which  the  dimensions  were  heuristic- 
ally  rotated  to  increase  their  interpretability.  This  analysis  did  not  result 
In  axis  orientations  that  were  more  interpretable  than  those  in  the  unrotated 
solution.  Because  the  unrotated  orientation  is  based  on  a  statistical  criterion 
(minimizing  the  inter-dimensional  correlations)  whereas  the  rotated  orientation 
was  based  only  on  a  heuristic  approach,  the  unrotated  placement  of  the  dimen¬ 
sions  is  used  in  the  interpretations  below. 


3.3.1  Interpretation  of  Dimension  One:  Degree  of  Uncertainty 

This  dimension  was  interpreted  as  referring  to  the  degree  of  uncer¬ 
tainty  in  the  information  associated  with  the  decision  function.  At  one  extreme 
of  the  dimension  were  the  two  signal  processing  decision  functions  —  Determine 
Signal  is  a  Valid  Contact,  and  Classify  Signal.  In  both  of  these  cases,  the 
inputs  to  the  TACCO's  decision  —acoustic  signals  —  are  well  defined  and 
precise.  The  uncertainty  in  these  decision  functions  lies  principally  in  the 
interpretation  or  analysis  of  these  signals.  At  the  other  end  of  the  dimension 
there  is  a  cluster  of  decision  function  relating  to  search  patterns  —  Extending 
the  Pattern,  Constructing  the  Pattern,  Constructing  the  Monitoring  Pattern,  and 
Compensating  for  in-situ  Propagation  Conditions.  There  are  many  inputs  to  these 
decisions  and  each  of  these  inputs  may  contain  a  great  deal  of  uncertainty  or 
"noise."  Also,  these  decision  functions  are  concerned  with  negative 
information,  that  is,  information  which  is  gained  when  something  is  observed 
not  to  occur.  For  example,  the  failure  to  gain  a  contact  from  certain  sensors 
provides  negative  information  on  the  target's  location.  There  is  a  great  deal 
of  uncertainty  associated  with  incorporating  negative  information  into  a 
decision.  Thus,  the  decision  functions  on  either  end  of  this  dimension  repre¬ 
sent  extremes  in  the  degree  of  uncertainty  in  the  inputs  to  the  decision. 

The  order  of  the  decisions  between  these  extremes  also  supports  this 
interpretation  of  the  dimension.  Highest  on  the  dimension  after  the  signal  pro¬ 
cessing  functions  are  (in  order)  Coordinate  Hand-off,  Determine  Weapon  and 
Setting,  and  Determine  Aircraft  Weapon  Launch  Position.  These  three  decision 
functions  all  involve  little  input  uncertainty.  In  a  hand-off  the  number, 
types,  and  settings  of  the  deployed  sonobuoys  are  all  known  precisely,  and  their 
locations  are  known  to  at  least  some  degree.  Since  there  currently  is  only  one 
available  weapon  with  two  possible  settings  on  the  P-3C,  there  is  also  little 
input  uncertainty  in  determining  the  attack  weapon  and  its  setting.  The 
aircraft  weapon  launch  position  involves  more  uncertainty  because  it  must  be 
calculated  on  the  basis  of  estimates  of  the  target’s  present  and  future  course, 
speed  and  location.  These  estimates  always  contain  an  element  of  uncertainty  since 
they  are  predictions  of  future  action. 
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Near  the  zero  point  on  the  dimension  is  the  Managing  Equipment  and 
Stores  decision  function.  This  decision  has  two  inputs,  one  of  which  (the 
stores  remaining)  is  somewhat  certain  and  the  other  of  which  (possible  future 
needs  for  these  stores)  is  highly  uncertain  (and  thereby  subject  to 
over-estimation). 

The  remaining  decision  functions  are  located  in  a  cluster  near  the 
negative  extreme  of  the  dimension.  All  of  these  relate  in  some  degree  to  target 
movement.  Anticipating  Target  Movement  explicitly  involves  predicting  the 
future  location,  course,  depth,  and  speed  of  the  target  using  (as  inputs)  the 
present  estimate  of  the  target's  location,  course,  depth,  and  speed.  Because 
they  are  estimates,  these  inputs  are  always  subject  to  some  degree  of  uncertain¬ 
ty  (as  are  all  appraisals  of  enemy  intent).  These  same  inputs  are  used  in  two 
other  decision  functions  in  the  cluster  —  Gaining  Attack  Criteria  and 
Determining  a  Target  Fix.  The  fourth  decision  function  in  this  group  is 
Adjusting  the  Sensor  Pattern  to  Sensor  Failure.  There  are  several  inputs  which 
the  TACCO  must  consider  in  making  this  decision.  The  characteristics  of  the 
current  sensor  pattern  —  the  geometry  of  the  pattern  and  the  setting,  type  and 
depth  of  existing  sensors  —  must  obviously  be  considered.  But  the  anticipated 
movement  of  the  target  is  also  an  input,  since  the  TACCO  may  desire  to  extend 
the  pattern  in  the  direction  of  future  target  movement  rather  than  just  replace 
a  faulty  sensor  in  the  area  from  which  the  target  is  moving.  Also,  the  likely 
remaining  lifespan  of  the  other  sensors  in  the  pattern  must  be  considered  in 
adjusting  the  pattern  to  the  failure  of  a  sensor.  These  last  two  types  of  input 
(anticipations  of  target  movement  and  estimates  of  remaining  sensor  lifespans) 
both  contain  a  good  deal  of  uncertainty  and  thus  justify  the  placement  of 
adjusting  the  Sensor  Pattern  to  Sensor  Failure  decision  function  with  the  others 
in  this  cluster. 

3.3.2  Interpretation  of  Dimension  Two:  Information  Processing  Load 

This  dimension  was  interpreted  as  representing  the  relative  information 
processing  load  the  decision  placed  upon  the  TACCO.  At  the  lower  extreme  of 
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this  dimension  are  the  two  signal  processing  decisions  (Determine  Signal  is  a 
Valid  Contact  and  Classify  Signal),  On  the  P-3C  these  decisions  are  primarily 
made  by  the  Sensor  Station  Operators  (SENSOs)  and  are  merely  accepted  or 
rejected  by  the  TACCO.  Thus,  they  present  a  very  light  information  processing 
load  for  the  TACCO.  At  the  other  extreme  are  five  decision  functions  —  Gaining 
Attack  Criteria,  Anticipating  Target  Movement,  Determining  Target  Fix, 
Determining  Aircraft  Weapon  Launch  Position,  and  Determining  Weapon  and  Setting 
for  Attack  —  which  must  be  made  in  short  periods  of  time  and  require  the 
integration  of  Information  of  many  types  of  information  from  many  sources.  For 
all  these  decisions  information  from  the  different  sensors  and  different  sensor 
types  must  be  correlated  and  compared  to  obtain  estimates  of  the  targets' 
location,  course,  depth,  and  speed.  Also,  information  on  the  sensor  types  and 
sensor,  capabilities  must  be  considered  together  with  environmental  information 
to  construct  assessments  of  the  likely  error  or  uncertainty  in  those  estimates. 
All  of  these  five  decision  functions  thus  place  a  heavy  information  processing 
load  on  the  TACCO. 

The  remaining  decision  functions  on  this  dimension  fall  near  each  other 
in  a  cluster  with  its  center  near  the  origin.  These  seven  decision  functions  — 
Coordinating  Handoff,  Extending  Sensor  Pattern,  Constructing  Sensor  Monitoring 
Pattern,  Creating  Sensor  Patterns,  Compensating  for  in  situ  Propagation 
Conditions,  Adjusting  Sensor  Pattern  and  Managing  Equipment/Stores  --  represent 
decisions  with  multiple  and  complex  inputs.  They  are,  however,  less  time 
constrained  than  those  decisions  at  the  positive  end  of  this  dimension.  In  most 
cases,  the  same  Information  must  be  processed  by  the  TACCO  in  these  decisions  as 
in  the  decision  functions  placed  at  the  high  end  of  the  dimension.  Environmental 
conditions,  sensor  capabilities  and  placements,  and  target  location  estimates 
must  be  considered  in  some  or  all  of  the  decisions  in  this  group,  and  multi-  and 
cross-sensor  Inputs  must  also  be  compared  and  correlated.  But  because  the  deci¬ 
sions  In  this  cluster  do  not  require  as  rapid  solutions,  the  TACCO  has  more  time 
to  process  this  Information.  Thus,  the  relative  Information  processing  load  is 
reduced. 
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This  interpretation  of  dimension  two  is  subject  to  some  degree  of  inde¬ 
pendent  validation  by  comparing  it  to  the  ranking  associated  with  the  judged 
criterion  of  workload.  Comparing  the  order  of  the  decision  functions  on  this 
dimension  with  their  order  in  the  average  ranking  for  TACCO  workload  in  Table  2-7, 
a  rank  order  correlation  of  .8  was  obtained.  This  high  correlation  supports  the 
interpreation  of  dimension  two  as  an  information  processing  load  dimension. 

3.3.3  Interpretation  of  Dimension  Three:  Complexity  of  Alternative  Structure 

Dimension  three  proved  the  most  difficult  to  interpret  and  its  analysis 
here  should  be  considered  as  only  tentative.  The  third  dimension  was  finally 
concluded  to  represent  the  relative  complexity  of  the  alternative  structure  of 
the  decision  function.  Complexity  of  the  alternative  structure  refers  to  the 
number  of  alternative  approaches  the  TACCO  has  toward  performing  the  decision 
function.  At  the  postive  extreme  of  this  dimension  were  three  decision  func¬ 
tions  which  can  be  approached  in  a  relatively  straight  forward  manner  — 

Managing  Equipment  and  Stores,  Coordinating  the  Handoff,  and  Determining  Weapon 
and  Setting  for  Attack.  Each  of  these  decision  functions  can  be  performed  by 
standard  procedures  and  do  not  present  a  large  number  of  alternative  choices  to 
the  TACCO.  In  coordinating  a  handoff,  for  example,  the  information  which  the 
TACCO  mist  manage  and  transmit  to  the  relief  platform  is  well  defined  and  there 
are  only  a  few  different  ways  in  which  he  may  do  so. 

The  placement  of  the  Adjusting  Sensor  Pattern  to  Sensor  Failure  deci¬ 
sion  function  with  this  group  is  somewhat  anomalous,  as  there  are  many  alternate 
ways  in  which  this  decision  function  may  be  approached.  If  the  TACCOs  were  con¬ 
sidering  it  as  a  straight  sensor  replacement  decision,  however,  its  location  on 
this  dimension  is  more  understandable. 

On  the  opposite  end  of  this  dimension  are  five  decision  functions 
relating  to  signal  processing  and  target  movement— Determi ni ng  Target  Fix, 
Anticipating  Target  Movement,  Classifying  a  Signal,  Determining  the  Signal  is  a 
Valid  Contact,  and  Gaining  Attack  Criteria.  Each  of  these  decision  functions 
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has  many  complex  inputs  and  can  be  approached  in  a  variety  of  different  ways. 

In  determining  a  target  fix,  for  example,  the  TACCO  may  rely  on  information  from 
any  combination  of  sensors  (of  the  same  or  of  differing  types)  to  determine  the 
precise  location  of  the  submarine.  The  number  of  possible  locations  for  the 
target  is  theoretically  infinite  and  even  in  practice  it  is  very  large.  Thus, 
the  TACCO  may  take  any  of  a  variety  of  approaches  to  select  from  among  the  large 
and  complex  set  of  alternatives  in  these  decisions.  The  remaining  five  deci¬ 
sion  functions  on  this  dimension— Determining  the  Aircraft  Weapon  Launch 
Position,  Compensating  for  in-situ  Propagation  Conditions,  Creating  a  Sensor 
Pattern,  Constructing  a  Sensor  Monitoring  Pattern,  and  Extending  the  Sensor 
Pattern  —  are  all  located  in  a  cluster  near  the  zero-point  of  this  dimension. 
These  all  seem  to  represent  decisions  which  have  a  moderately  large  set  of 
alternatives,  but  not  as  large  or  complex  a  set  as  the  signal  processing  or 
target  movement  decisions. 

An  alternative  interpretation  of  this  dimension  is  to  dichotomize  the 
decision  functions  into  tactical  decisions  and  target  management  decisions.  The 
tactical  decisions  are  Manage  Equipment/Stores,  Coordinate  Handoff,  Determine 
Weapon  and  Setting  for  Attack,  Adjust  Sensor  Pattern  to  Sensor  Failure, 

Determine  Aircraft  Weapon  Launch  Position,  Compensate  for  in-situ  Propagation 
Conditions,  Create  Sensor  Pattern,  Construct  Sensor  Monitoring  Pattern  and 
Extend  Sensor  Pattern.  All  these  decisions  involve  choices  made  by  the  TACCO 
concerning  the  selection  and/or  use  of  specific  tactics  to  gain  contact  with  or 
prosecute  the  submarine.  These  decisions  are  all  located  on  the  positive  end  of 
the  dimension  with  the  exception  of  Extend  Sensor  Pattern  which  has  a  value  near 
zero  (-.02). 

The  remaining  decision  functions  (Gain  Attack  Criteria,  Determine 
Signal  as  a  Valid  Contact,  Classify  Signal  Anticipate  Target  Movement,  Determine 
Target  Fix)  are  all  located  on  the  negative  end  of  the  dimension,  substantially 
away  from  the  zero  point.  These  decisions  do  not  concern  the  selection  or 
application  of  tactics  but  instead  deal  with  interpretation  and  analysis  of 


information  that  is  received  about  a  specific  target  as  a  result  of  previous 
tactical  actions.  These  decisions  affect  the  tactical  decisions  on  the  other 
half  of  the  dimension,  both  in  terms  of  allowing  new  tactics  to  be  employed  (as, 
for  example,  attainment  of  attack  criteria  permits  the  attack  tactics  selection 
to  begin)  and  in  terms  of  providing  feedback  on  the  effectiveness  of  previous 
tactics  (as,  for  example,  verifying  a  contact  indicates  that  the  current  sensor 
pattern  has  been  effective  in  achieving  a  contact  from  a  target  of  interest). 


4.  PRIORITIZATION  OF  AIR  ASW  DECISION  SITUATIONS 


The  three  dimensions  constructed  by  the  Multidimensional  Scaling  analy¬ 
sis  of  the  sorting  data  represent  a  geometric  model  of  experienced  TACCOs'  con¬ 
ceptual  framework  for  Air  ASW  decisions.  Since  these  three  dimensions  represent 
the  basic  underlying  characteristics  which  experienced  TACCOs  use  to  think  about 
Air  ASW  decision  making,  other  subjective  characteristics  should  be  representable 
as  combinations  of  these  underlying  features.  Priority  or  importance  is  one 
such  characteristic.  This  section  describes  the  procedures  used  to  identify  the 
particular  combination  of  these  three  dimensions  which  the  TACCOs  interviewed  at 
Moffett  Field  were  implicitly  using  when  they  ranked  the  decision  functions 
according  to  their  importance  in  the  mission.  Also  presented  is  the  way  in  which 
the  reconstructed  combination  rules  or  priority  functions  were  used  to  numeri¬ 
cally  prioritize  the  six  Air  ASW  Decision  situations  for  decision  aid  design. 

The  basic  technique  used  to  determine  the  TACCOs'  priority  functions  is 
termed  Priority  Mapping,  and  has  been  developed  specifically  for  this  effort.  It 
is  related  to  the  psychometric  technique  known  as  Preference  Mapping  (Carrol  and 
Chang  1967)  and  is  based  on  Coombs  (1950)  and  Bennett  and  Hays  (1960)  notion  of 
Unfolding  Analysis.  Appendix  F  provides  a  review  of  Unfolding  Analysis  and  its 
relation  to  Priority  Mapping.  Briefly,  Priority  Mapping  is  a  means  of  "embedding" 
an  ordinal  priority  scale  of  a  set  of  items  into  a  multidimensional  model  of  the 
interrelationships  among  those  items.  Normally,  the  multidimensional  model  is  a 
the  result  of  an  MDS  analysis  of  a  dissimilarity  measure  on  a  set  of  items,  and 
the  priority  scale  is  a  ranking  of  the  items  produced  by  one  or  more  individuals 
according  to  some  definition  or  criterion  of  priority.  The  scale  is  embedded 
into  the  multidimensional  model  by  expressing  the  scale  as  some  function  of  the 
dimensions  in  the  model. 
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Unfolding  Analysis  determines  which  among  four  broad  classes  of  func¬ 
tions  provides  the  best  representation  of  the  ordinal  scale  in  terms  of  the  multi¬ 
dimensional  model  dimensions.  These  four  classes  of  functions  correspond  to  four 
increasingly  complex  multidimensional  models  of  priority.  In  the  simplest  model, 
called  the  vector  model,  priority  is  modeled  as  the  order  of  the  projections  of 
the  items  in  the  multidimensional  space  onto  a  vector  through  the  space.  The 
next  simplest  model  is  called  the  distance  model,  and  in  it  priority  is  modeled 
as  the  order  of  the  distances  of  the  items  in  the  multidimensional  space  from 
some  "ideal “  point  representing  the  highest  possible  (or  lowest  possible) 
priority.  The  third  model  is  called  the  weighted  distance  model.  It  is  iden¬ 
tical  to  the  second  (distance)  model  except  that  it  allows  the  distance  from 
the  ideal  point  along  some  dimensions  to  contribute  more  to  priority  than  the 
distance  along  others.  Finally,  the  fourth  and  most  complex  model  is  termed  the 
general  distance  model.  It  is  identical  to  the  weighted  distance  model  except 
that  it  also  allows  the  dimensional  axes  to  be  rotated  prior  to  differential 
weighting.  These  four  models  are  discussed  in  greater  detail  in  Appendix  F. 
Section  4.1  summarizes  the  Priority  Mapping  results  for  the  importance  rankings 
collected  from  the  TACCOs  at  Moffett  Field.  Section  4.2  describes  the  manner  in 
which  these  results  were  used  to  prioritize  the  six  ASW  decision  situations. 

4.1  UNFOLDING  ANALYSIS  AND  PRIORITY  MAPPING  OF  DECISION  FUNCTIONS 

Separate  Unfolding  Analysis  procedures  were  conducted  for  the  two 
rankings  of  decision  functions  by  importance.  The  first  analysis  considered  the 
rankings  by  importance  in  a  mission  where  the  objective  was  an  attack  on  the 
submarine.  The  average  ranking  according  to  this  criterion  is  given  in  Table 
2-4,  and  the  rankings  by  each  of  the  individual  TACCOs  are  given  in  Appendix  D. 

The  second  analysis  considered  the  ranking  by  importance  in  a  mission  where  the 
goal  was  surveillance  of  the  submarine.  The  average  ranking  according  to  this 
criterion  is  given  in  Table  2-5,  and  the  rankings  by  each  of  the  individual 
TACCOs  are  given  in  Appendix  D.  The  Priority  Mapping  analysis  was  accomplished 
by  using  the  PREFMAP  program  developed  by  Carroll  and  Chang  (1967,  1977),  which 
performs  the  procedures  described  in  Appendix  F.  PREFMAP,  like  the  MDS  programs 


? 

J 


described  in  Section  3.2,  is  part  of  the  MDSX  package  of  Multidimensional 
Scaling  programs  (Coxon  et  at.  1977)  available  on  the  Wharton  School's  DEC- 10 
Computer. 


PREFMAP  performs  Unfolding  Analysis  in  two  parallel  manners.  First,  it 
conducts  a  separate  analysis  of  each  individual  ranking  against  a  multidimensional 
representation  of  the  ranked  stimuli  provided  by  the  user.  Second,  it  performs 
an  identical  analysis  on  the  aggregated  (average)  ranking  of  all  the 
individuals.  In  the  discussions  of  both  of  the  Unfolding  Analysis  procedures 
below  only  the  results  of  the  aggregate  analysis  are  considered  because  they 
were  the  ones  that  were  used  in  the  prioritization  of  the  six  decision 
situations. 

4.1.1  Decision  Function  Prioritization  for  Attack  Missions 

The  analyses  of  the  decision  function  rankings  for  attack  missions 
concluded  that  the  most  compex  of  the  four  models  of  priority--general  distance 
model— was  required  to  represent  this  criterion  (i.e.  importance  in  a  mission 
with  attack)  as  a  function  of  the  three  dimensions  discussed  in  Section  3.  The 
detailed  calculations  in  the  Priority  Mapping  for  this  criterion  are  given  in 
Appendix  G.  The  result  of  the  analyses  is  a  formula  for  determining  the 
priority  score,  PwA^djh  any  decision  function,  dj,  in  an  attack  mission  from 
its  coordinates  in  the  multidimensional  decision  space  (Figure  3-1)  alone.  This 
formula  is  also  given  in  Appendix  G.  The  numerical  priority  scores  that  were 
generated  for  the  fourteen  decision  functions  by  the  formula  PwA(dj)  are  given 
in  Table  4-1. 

The  values  in  Table  4-1  represent  the  final  prioritization  of  the  deci¬ 
sion  functions  in  a  mission  with  attack.  It  should  be  noted  that  in  Table  4-1 
lower  values  indicate  higher  priority,  and  higher  values  indicate  lower 
priority.  The  interviewees  were  asked  to  assign  the  most  important  decision  the 
lowest  rank  (i.e.  1),  and  the  least  important  decision  the  highest  rank  (i.e. 

14),  and  PREFMAP  merely  maintains  this  directionality  in  its  computations. 


Table  4-1.  Decision  Function  Prioritization  For  Mission  With  Attack  On  Submarine 


PRIORITY  RANK 

DECISION  FUNCTION 

PRIORITY  SCORE 

1 

DETERMINE  SIGNAL  IS  VALIO  CONTACT 

-.2219 

(HIGHER) 

2 

CLASSIFY  SIGNAL 

-.2208 

3 

COORDINATE  HAND^FF 

-.1553 

4 

DETERMINE  TARGET  FIX 

-.1439 

5 

ANTICIPATE  TARGET  MOVEMENT 

-.1186 

6 

CONSTRUCT  SENSOR  MONITORING  PATTERN 

-.0581 

7 

COMPENSATE  FOR  IN*SITU  PROPAGATION 

-.0575 

PRIORITY 

CONDITIONS 

8 

CREATE  SENSOR  PATTERN 

-.0350 

9 

EXTEND  SENSOR  PATTERN 

-.0293 

10 

MANAGE  EQUIPMENT/STORES 

.0032 

11 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

.0167 

12 

GAIN  ATTACK  CRITERIA 

.2098 

13 

DETERMINE  AIRCRAFT  WEAPON  LAUNCH  POSITION 

.4050 

14 

DETERMINE  WEAPON  AND  SETTING  FOR  ATTACK 

.4322 

(LOWER) 

4>1*2  Decision  Function  Prioritization  in  Surveillance  Missions 

A  parallel  analysis  was  conducted  for  the  rankings  by  importance  in  a 
surveillance  mission— one  without  an  attack  segment.  In  this  case  also  it  was 
found  that  the  generalized  distance  model  was  required  to  describe  the  TACCOs' 
rankings  of  priority.  The  Priority  Mapping  resulted  in  a  formula  for  determining 
the  priority  score,  PwOA(dj).  of  a  decision,  dj,  in  a  surveillance  mission  from 
Its  location  on  the  three  dimensional  space  shown  in  Figure  3-1.  This  formula, 
and  the  details  of  the  Unfolding  Analysis  and  Priority  Mapping  for  this  cri¬ 
terion  are  also  given  in  Appendix  G.  The  priority  scores  for  the  fourteen  deci¬ 
sions  generated  by  the  formula  PwOA^j)  are  given  in  Table  4-2.  These  values 
represent  the  priorities  of  the  decision  functions  in  a  surveillance  mission. 
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As  described  above,  smaller  values  indicate  higher  priority,  and  larger  values 
cate  lower  priority. 

Table  4-2.  Decision  Function  Prioritization  For  Mission  Without  Attack  On  Submarine 


PRIORITY 

RANK 


NAME 

PRIORITY 

SCORE 

GAIN  ATTACK  CRITERIA 

-.21129 

(HIGHER) 

DETERMINE  SIGNAL  IS  VALID  CONTACT 

-.186603 

CLASSIFY  SIGNAL 

-.17522 

DETERMINE  AIRCRAFT  WEAPON  LAUNCH  POSITION 

-.15907 

DETERMINE  TARGET  FIX 

-.14836 

ANTICIPATE  TARGET  MOVEMENT 

-.08897 

DETERMINE  WEAPON  ANO  SETTING  FOR  ATTACK 

-.02783 

PRIORITY 

COMPENSATE  FOR  IN-SITU  PROPAGATION  CONDITIONS 

.03061 

CREATE  SENSOR  PATTERN 

.05525 

CREATE  SENSOR  MONITORING  PATTERN 

.06229 

EXTEND  SENSOR  PATTERN 

.08192 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

.15309 

MANAGE  EQUIPMENT/STORES 

.15867 

COORDINATE  HANC-OFF 

.45690 

(LOWER) 

•2  COMBINING  PRIORITY  SCORES  ACROSS  DECISION  SITUATIONS 


Tables  4-1  and  4-2  present  the  priority  scores  that  were  constructed  for 
the  fourteen  decision  functions  in  two  types  of  missions  --  missions  with  and 
without  an  attack  on  the  submarine  being  prosecuted.  These  decision  function 
prioritizations  still  remain  to  be  translated  into  decision  Htvation 
prioritizations.  This  is  accomplished  in  a  three  step  procedure.  In  the  first 
step,  the  decision  situations  are  broken  down  into  their  constituent  decision 
functions.  In  the  second  step,  the  priority  scores  for  all  the  decision  func¬ 
tions  which  comprise  each  situation  are  combined  to  produce  a  raw  priority, 
score.  And  in  the  third  step,  the  raw  priority  score  for  each  situation  is  nor¬ 
malized  to  compensate  for  the  differing  number  of  functions  used  to  represent 
each  situation.  Each  of  these  steps  is  described  in  greater  detail  below. 


4.2.1  Relating  Decision  Functions  to  Decision  Situations 

Initially,  the  decision  functions  which  comprise  each  of  the  six  deci¬ 
sion  situations  were  identified.  An  initial  assessment  of  the  relationship  be¬ 
tween  the  decision  functions  was  made  in  the  preceding  effort  (see  Zachary  1980, 
Section  2.).  The  breakdown  of  decision  situations  into  constituent  decision 
functions  presented  there  [ibid:  Table  2-2)  was  refined  through  discussions  with 
experienced  Naval  ASW  personnel  at  the  Naval  Air  Development  Center  (NADC).  As 
a  result  of  these  discussions,  the  decision  functions  (from  among  the  fourteen 
in  the  stimulus  set)  involved  in  each  of  the  six  decision  situations  (On-Station 
Search,  Contact  Classification/Verification,  Localization,  Surveillance 
Tracking,  Lost  Contact  Reacquistion,  and  Attack  Planning)  were  identified.  The 
relationships  between  the  decision  functions  and  situations  are  given  in  Table 
4-3.  It  should  be  noted  that  some  of  the  decision  functions  are  included  in 
several  of  the  decision  situations.  This  reflects  the  conclusions  of  the  pre¬ 
vious  effort  that  the  decision  situations  reflect  contexts  in  which  specific 
decisions  are  made,  and  that  the  same  decision  function  may  be  performed  quite 
differently  in  different  mission  contexts. 

4.2.2  Creating  Raw  Priority  Scores  for  the  Decision  Situations 

After  the  constituent  decision  functions  of  each  decision  situation 
were  identified,  raw  priority  scores  for  the  decision  situations  were 
constructed.  This  was  done  by  summing  the  priority  scores  of  the  constituent 
decision  functions  for  each  situation.  Since  each  decision  function,  j,  had  two 
priority  scores  —  PyA(dj)  and  pW0A(dj)  two  corresponding  raw  priority  scores 
were  constructed  for  each  situation.  The  first  represented  the  situations 
priority  in  a  mission  whose  goal  was  attack  and  destruction  of  the  hostile  sub¬ 
marine,  and  was  computed  as 

PwA(Di)  =*  I  PWA(dj) 
djE  D-j 
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Table  4-3.  Decision  Function  Composition  Of  Six  Air  ASW  Decision  Situations 


DECISION  SITUATION 

CONSTITUENT  DECISION  FUNCTIONS 

ON-STATION  SEARCH 

CREATE  SENSOR  PATTERN 

MANAGE  EQUIPMENT/STORES 

EXTEND  SENSOR  PATTERN 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

CONSTRUCT  SENSOR  MONITORING  PATTERN 
COMPENSATE  FOR  IN-SITU  PROPAGATION  CONDITIONS 

CONTACT  CLASSIFICATION/VERIFICATION 

CLASSIFY  SIGNAL 

DETERMINE  SIGNAL  IS  A  VALID  CONTACT 

LOCALIZATION 

MANAGE  EQUIPMENT/STORES 

ANTICIPATE  TARGET  MOVEMENT 

EXTEND  SENSOR  PATTERN 

GAIN  ATTACK  CRITERIA 

DETERMINE  TARGET  FIX 

COORDINATE  HAND-OFF 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

CONSTRUCT  SENSOR  MONITORING  PATTERN 
COMPENSATE  FOR  IN-SITU  PROPAGATION  CONDITIONS 

SURVEILLANCE  TRACKING 

CREATE  SENSOR  PATTERN 

MANAGE  EQUIPMENT/STORES 

ANTICIPATE  TARGET  MOVEMENT 

EXTEND  SENSOR  PATTERN 

DETERMINE  TARGET  FIX 

COORDINATE  HAND-OFF 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

CONSTRUCT  SENSOR  MONITORING  PATTERN 
COMPENSATE  FOR  IN-SITU  PROPAGATION  CONDITIONS 

ATTACK  PLANNING 

CREATE  SENSOR  PATTERN 

MANAGE  EQUIPMENT/STORES 

ANTICIPATE  TARGET  MOVEMENT 

EXTEND  SENSOR  PATTERN 

GAIN  ATTACK  CRITERIA 

DETERMINE  TARGET  FIX 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

DETERMINE  WEAPON  AND  SETTING  FOR  ATTACX 
CONSTRUCT  SENSOR  MONITORING  PATTERN 

DETERMINE  AIRCRAFT  WEAPON  LAUNCH  POSITION 
COMPENSATE  FOR  IN-SITU  PROPAGATION  CONDITIONS 

LOST  CONTACT  REACQUISITION 

CREATE  SENSOR  PATTERN 

MANAGE  EQUIPMENT/STORES 

ANTICIPATE  TARGET  MOVEMENT 

EXTEND  SENSOR  PATTERN 

COORDINATE  HAND-OFF 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

CONSTRUCT  SENSOR  MONITORING  PATTERN  . 
COMPENSATE  FOR  IN-SITU  PROPAGATION  CONDITIONS 
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where  Di  is  decision  situation  i,  dj  is  decision  function  j,  and  is  as  pre¬ 
viously  defined.  The  raw  decision  situation  priority  scores  for  a  mission  with 
attack  are  given  in  Table  4-4. 


Table  4-4.  Decision  Situation  Prioritization  For  Mission  With  Attack 


RANK 

SITUATION 

RAW  PRIORITY 
SCORE 

NORMALIZED 
PRIORITY  SCORE 

1 

CONTACT  CLASSIFICATION/ 

-.36 

-.181 

VERIFICATION 

2 

ATTACK  PLANNING 

-.09 

-.008 

3 

LOCALIZATION 

+.49 

+.055 

4 

SURVEILLANCE  TRACKING 

+.76 

+.084 

5 

LOST  CONTACT  REACQUISITION 

+.71 

+.089 

e 

ON-STATION  SEARCH 

+.54 

+.090 

The  second  priority  score  for  each  decision  situation  represented  its 
priority  in  a  mission  whose  goal  was  surveillance  of  the  hostile  submarine,  and 
was  computed  as 


PwOA(°i)  a  z  pW0A(dj) 
dj  e  Di 

where  Di,  dj,  and  Pyg^  are  as  previously  defined.  The  raw  decision  situation 
priority  scores  for  a  mission  without  attack  are  given  Table  4-5. 


4-8 


Table  4-5.  Decision  Situation  Prioritization  For  Mission  Without  Attack 


RANK 

SITUATION 

RAW  PRIORITY 
SCORE 

NORMALIZED 

PRIORITY 

SCORE 

1 

CONTACT  CLASSIFICATION/ 

-.44 

-.221 

VERIFICATION 

2 

SURVEILLANCE  TRACKING 

-.58 

-.064 

3 

LOST  CONTACT  REACQUISITION 

-.43 

-.054 

4 

LOCALIZATION 

-.33 

-.037 

5 

ON-STATION  SEARCH 

-.16 

-.027 

6 

ATTACK  PLANNING 

+.62 

+.056 

4.2.3  Normalizing  the  Raw  Scores 

The  raw  priority  score  assigned  to  each  decision  situation  is 
influenced  by  two  factors  —  the  priority  scores  of  the  constituent  decision 
functions  and  the  number  of  constituent  decision  functions.  This  second  factor 
has  a  substantial  effect  on  the  raw  score,  because  the  greater  the  number  of 
constituent  functions,  the  larger  the  raw  score.  Unfortunately  the  effect  of 
this  factor  is  primarily  to  confound  the  effect  of  the  first  which  is  the  prin¬ 
cipal  concern  in  this  prioritization  procedure. 


When  the  stimulus  set  for  the  MDS  procedure  was  defined,  it  was  not 
intended  that  it  include  every  conceivable  decision  function  which  the  TACCO 
performs  in  an  Air  ASW  mission.  Rather,  it  was  simply  constructed  to  include  a 
representative  sample  of  the  decision  functions  which  may  arise  in  the  course  of 
a  mission.  The  fourteen  decision  functions  which  were  chosen  are  not  evenly 
distributed  across  the  six  decision  situations,  as  can  be  seen  from  Table  4-3. 
One  situation  (Contact  Classification/Verification)  was  represented  by  as  few  as 
two  functions,  while  another  (Attack  Planning)  was  represented  by  as  many  as 
eleven.  The  distribution  of  the  sample  of  decision  functions  across  the  deci¬ 
sion  situations  does  not  reflect  the  distribution  of  all  decision  functions 
over  the  situations  and,  therefore,  does  not  meaningfully  Impact  on  situational 
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priority.  The  number  of  decision  functions  in  a  decision  situation  affects  the 
workload  of  the  TACCO  in  that  situation,  but  should  not  affect  the  priority  of 
that  decision  situation  in  the  mission.  If  it  did,  then  the  highest  priority 
situation  would  be,  by  definition,  the  situation  in  which  the  TACCO  is  the 
busiest. 

It  was  therefore  necessary  to  control  for  the  effect  of  the  differing 
number  of  decision  functions  contributed  to  the  raw  priority  score  for  each 
situation.  This  was  done  by  normalizing  the  raw  score  --  dividing  it  by  the 
number  of  terms  in  the  sum  from  which  it  was  computed.  That  is,  the  raw  score 
was  divided  by  the  number  of  decision  functions  which  comprise  the  decision 
situation.  This  normalization  procedure  was  performed  for  both  sets  of  raw 
scores.  The  normalized  priority  scores  of  each  situation  for  the  mission  with 
attack  are  given  next  to  the  raw  scores  in  Table  4-4.  The  normalized  scores  for 
the  mission  without  attack  are  given  next  to  the  raw  scores  for  each  situation 
in  Table  4-5. 

4.2.4  Two  Priorizations  of  Air  ASW  Decision  Situations 

The  normalized  scores  for  the  decision  situations  in  each  type  of 
mission  were  then  used  to  rank  the  situations  according  to  their  overall 
priority.  The  orders  in  which  the  situations  are  shown  in  Table  4-4  and  4-5 
reflect  these  rankings.  In  a  mission  with  the  goal  of  attacking  the  submarine 
the  final  prioritization  of  the  decision  situations  is: 

•  Contact  Classification/Verification 

•  Attack  Planning 

t  Localization 

•  Surveillance  Tracking 

•  Lost  Contact  Reacquisition 

•  On-Station  Search 


This  order  is  intuitively  attractive,  since  the  Attack  Planning  situation  is 
placed  near  the  top,  while  Surveillance  Tracking  (which  presumably  would  not 
occur  in  this  type  of  mission)  is  near  the  bottom.  In  numerical  terms,  the  bot¬ 
tom  three  situations  are  only  separated  by  .006,  indicating  that  they  are  equally 
low  in  priority  (see  Table  4-4). 

In  a  mission  with  the  goal  of  surveillance  of  the  submarine,  the  final 
prlorlzation  of  the  decision  situations  is: 

e  Contact  Classification/Verification 

e  Surveillance  Tracking 

•  Lost  Contact  Reacquisition 

•  Localization 

•  On-Station  Search 

•  Attack  Planning 

This  order  Is  also  intuitively  acceptable,  with  Attack  Planning  (which  would  not 
occur  in  this  type  of  mission)  at  the  bottom  and  Surveillance  Tracking  near  the 
top. 


5.  DECISION  SITUATION  PRIORITIZATION  AND  DECISION  AID  DESIGN 


The  primary  purpose  for  prioritizing  the  six  decision  situations  is  to 
determine  the  order  in  which  decision  aiding  technology  should  be  applied  to 
them.  In  this  section,  the  highest  priority  decision  situations  are  analyzed 
and  high  level  decision  aid  designs  are  constructed  for  them.  Two  specific 
situations  will  be  considered.  Contact  Classification/Verification  (which  had 
the  highest  score  in  both  types  of  mission),  and  Attack  Planning  (which  had  the 
second  highest  priority  for  the  mission  with  attack). 

5.1  CONTACT  CLASSIFICATION/VERIFICATION:  AN  OPTIMAL  MODE  SELECTION 
DECISION  Alff 

The  Contact  Classification/Verification  situation  was  ranked  first  in 
the  priority  scales  for  both  types  of  mission.  Since  this  situation  was  repre¬ 
sented  by  only  two  decision  functions  in  the  sample  of  fourteen,  it  is  possible 
that  this  result  is  partially  an  artifact.  It  was  clear  from  the  interviews  at 
Moffett  Field,  however,  that  the  TACCOs  felt  this  situation  to  be  crucial  to  the 
ASW  mission,  for  two  specific  reasons.  First,  they  felt  that  the  entire  prose¬ 
cution  phase  of  the  mission  was  dependent  upon  the  correctness  of  the  decisions  in 
this  situation.  Second,  they  felt  that  there  was  very  little  room  for  error  in 
the  decisions  in  this  situation.  These  comments,  while  explaining  the  high 
priority  assigned  to  this  situation,  were  not  useful  in  determining  what  type  of 
decision  aid  is  needed  for  it.  Information  more  pertinent  to  this  end  was 
obtained  from  the  previous  analysis  and  description  of  the  Contact  Classification/ 
Verification  decision  situation  (see  Zachary  1980),  the  dimensional  interpre¬ 
tations,  and  the  additional  rankings  (urgency  and  workload)  collected  at  Moffett 
Field. 
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The  two  decision  functions  in  the  sample  of  fourteen  which  contribute 
to  this  decision  situation  (Classify  Signal  and  Determine  Signal  is  Valid  Contact) 
lay  at  or  near  an  extremum  of  each  of  the  three  dimensions  in  the  MDS  solution. 

On  the  first  dimension,  degree  of  uncertainty,  they  are  located  at  the  end 
representing  little  information  uncertainty.  The  situational  inputs  identified 
in  the  decision  situation  description  confirm  this.  The  primary  inputs  — 
acoustical  signals  —  are  tangible  and  lacking  in  uncertainty:  it  is  the 
processing  and  interpretation  of  these  signals  which  is  more  problematical. 

On  the  second  dimension,  information  processing  load,  the  two  decision 
functions  relevant  to  this  situation  were  located  near  the  end  representing  a 
low  processing  load.  This  is  presumably  because  the  major  part  of  the  decisions 
made  in  this  situation  are  made  by  the  sensor  station  operators  (SENSO)  rather 
than  the  TACCO  himself.*  The  workload  rankings  support  this  conclusion,  as  the 
two  decision  functions  for  this  situation  ranked  twelfth  and  thirteenth  in  the 
average  rankings  (Table  2-7). 

On  the  third  dimension,  complexity  of  alternative  structures,  these 
same  two  decision  functions  were  located  near  the  end  of  the  dimension  repre¬ 
senting  high  complexity  of  alternative  structure.  This  indicated  that  there  is 
a  large  and  complex  set  of  alternative  ways  in  which  the  TACCO  or  SENSOs  can 
approach  these  decisions.  The  placement  of  the  decision  functions  on  dimension 
one  (degree  of  uncertainty)  confirms  this.  It  was  concluded  from  the  placement 
on  dimension  one  that  the  primary  difficulty  in  these  decision  functions  was  in 
the  analysis  of  the  input  data,  rather  than  in  the  input  data  itself.  This 
aspect  of  the  decision  situation  provided  the  key  to  designing  a  decision  aid. 
Further  investigation  of  the  detailed  manner  in  which  alternative  ways  of  pro¬ 
cessing  accoustic  signals  helped  identify  a  key  problem  in  need  of  aiding. 

*At  least  in  the  P-3C,  the  aircraft  flown  by  the  TACCOs  interviewed  at 
Moffett  Field.  On  the  S-3A,  however,  the  TACCO  can  assume  more  of  the  workload 
in  this  situation. 


The  increasing  sophistication  of  the  on-board  acoustic  signal  processing 
equipment  (i.e.  the  current  AQA-7(V7)  system  and  the  soon- to-be-implemented 
PROTEUS  system)  allows  incoming  sensor  information  to  be  processed  in  a  wide 
variety  of  modes.  Many  of  these  modes  can  be  employed  simultaneously,  resulting 
in  a  large  number  of  possible  processing  mode  combinations.  The  TACCO  or  SENSO 
must  choose  a  specific  processing  mode,  but  currently  has  very  little  time  and/ 
or  information  on  which  to  base  this  choice.  Moreover,  a  choice  of  the  correct 
processing  mode  is  crucial  to  the  successful  Identification  and  classification 
of  a  signal.  Thus  a  decision  aid  is  needed  to  assist  the  TACCO  in  selecting  an 
optimal  processing  mode  for  the  signal  processing  equipment. 

This  optimal  mode  selection  decision  aid  will  make  suggestions  as  to 
processing-mode  selection  based  on  the  following  input  information: 

•  Current  environmental  conditions,  e.g.  propagation-loss 
profile,  bathythermal  conditions,  sea-state, 

•  Characteristic  frequencies  of  the  potential  targets  of 
interest, 

•  Type,  placement,  and  setting  of  currently  deployed  sensors,  and 

•  Stability  of  the  various  frequencies  of  the  potential 
targets  of  interest. 

The  aid  obtains  some  of  the  input  information  from  the  TACCO,  and  the  remainder 
from  other  computer  programs  or  databases  on  board  the  platform.  In  particular, 
the  characteristic  target  frequencies  and  the  stability  of  target  frequencies 
will  be  obtained  from  an  on-board  database  which  is  loaded  prior  to  the  start 
of  the  mission,  according  to  the  targets  identified  as  likely  to  be  encountered. 
The  location,  depth,  setting  and  types  of  deployed  sensors  will  be  obtained  from 
other  operational  programs  on  the  aircraft.  The  specific  targets  of  interest  to 
the  TACCO  will  be  obtained  directly  from  him,  as  will  be  the  current  environmen¬ 
tal  conditions.  The  aid  will  produce  a  single  output  from  those  inputs  —  a 
suggested  acoustical  signal -processing  mode. 
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The  design  for  such  an  aid  is  described  below  in  terms  of  the  taxonomy 
of  decision  aiding  techniques  created  by  Analytics  as  a  part  of  the  previous 
effort.  The  taxonomy  is  shown  here  as  Table  5-1.  The  specific  techniques 
needed  from  each  category  of  the  taxonomy  in  the  optimal  mode  selection  aid  are 
discussed  below. 

5.1.1  Outcome  Calculators 

As  discussed  in  our  previous  report  (Zachary  1980)  there  is  no  real-time 
process  which  underlies  the  decisions  made  in  the  Contact  Classification/Verifi¬ 
cation  situation.  Therefore,  no  outcome  calculator  is  needed  by  a  decision  aid 
for  this  situation. 

5.1.2  Value  Model 

Central  to  the  mode  selection  problem  is  a  set  of  tradeoffs  created  by 
uncertainties  about  environmental  conditions  and  by  differing  characteristics  of 
the  various  targets  of  interest.  For  example,  some  processing  modes  may  improve 
the  probability  of  identifying  some  targets  but  decrease  the  probability  of 
detecting  others.  Alternatively,  some  modes  may  improve  the  probability  of 
detecting  certain  types  of  targets  but  only  if  specific  environmental  conditions 
prevail.  A  value  model  is  thus  needed  to  establish  the  preferred  tradeoffs 
among  such  factors.  The  multiattribute  utility  model  (MAUM)  can  be  used  effec¬ 
tively  in  this  type  of  situation.  A  preference  value  or  utility  will  be  com¬ 
puted  for  each  specific  combination  of  processing  modes,  by  the  following  MAUM: 

U(i )  =  z  WjP,(j) 
j  e  T 

In  the  model,  U(1)  is  the  utility  of  mode  combination  i,  T  is  the  set  of  targets 
of  interest,  j  is  a  specific  type  of  target,  PjU)  is  the  probability  of  iden¬ 
tifying  a  signal  from  target  type  j  given  the  current  sensor  pattern  and  pro¬ 
cessing  mode  combination  i,  and  Wj  Is  a  weight  representing  the  relative 
Importance  of  detecting  a  target  of  type  j  over  all  other  target  types.  This 

t — X — \ 
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Table  5-1.  Taxonomy  of  Decision  Aiding  Techniques 


1. 

OUTCOME  CALCULATORS 

1.1 

Closed  Form  Analytic  Models 

1.2 

Probabilistic  Models 

1.3 

Deterministic  Simulations 

1.3.1  Mechanical 

1.3.2  Differential  Equation 

1.4 

Monte- Carlo  Simulations 

2. 

VALUE  MODEL 

2.1 

Multi-Attribute  Utility  Model  (MAUM) 

2.2 

Adaptively  Constructed  MAUM 

2.3 

Direct  Assignment  of  Utilities  to  Outcomes 

2.4 

Risk-Incorporating  Utility  Models 

2.5 

Non-Linear  Utility  Model 

3. 

DATA  CONTROL  TECHNIQUES 

3.1 

Automatic  Data  Aggregation 

3.2 

Data  Management  Techniques 

4. 

ANALYSIS  TECHNIQUES 

4.1 

Optimization  Techniques 

4.1.1  Linear  Programming 

4.1.2  Non-Linear  Programming 

4.1.3  Dynamic  Programming 

4.1.4  Fibbonaci  Search 

4.1.5  Response  Surface  Methodology 

4.2 

Artificial  Intelligence  Techniques 

4.2.1  Heuristic  Search 

4.2.2  Bayesian  Pattern  Recognition 

4.3 

Sensitivity  Analysis 

4.4 

Intra-Process  Analysis 

4.5 

Information  Processing  Algorithms 

4.6 

Status  Monitor  and  Alert 

4.7 

Statistical  Analysis 

4.7.1  Distribution  Comparison 

4.7.2  Regression-Correlation 

4.7.3  Discriminant  Analysis 

4.7.4  Bayesian  Updating 

5. 

DISPLAY/DATA  ENTRY  TECHNIQUES 

5.1 

Oisplay  Graphics 

5.2 

Interactive  Graphics 

5.3 

Windowing 

5.4 

Speech  Synthesis/ Recognition 

5.5 

Quickening 

6. 

HUMAN  JUDGMENT  AMPLIFYING/ 
REFINING  TECHNIQUES 

6.1 

Operator-Aided  Optimization 

6.2 

Adaptive  Predictions 

6.3 

Bayesian  Updating 

5-5 


MAUM  incorporates  the  tradeoffs  among  target  types  through  the  Wj  factors,  and 
incorporate  the  tradeoffs  among  alternative  environmental  states  implicitly 
through  the  ( j ) . 

5.1.3  Data  Control 

While  the  actual  processing  of  the  incoming  acoustical  signals  may 
require  sophisticated  data  control  methods,  the  mode  selection  problem  does  not 
consider  these  signals  per  se.  The  information  it  does  require  (described 
above)  can  be  obtained  and  managed  with  current  standard  techniques,  so  no  data 
control  techniques  are  needed  in  this  aid. 

5.1.4  Analysis 

At  the  very  heart  of  the  mode  selection  decision  is  an  optimization 
problem:  the  optimal  processing  mode  must  be  chosen  from  among  all  possible 
combinations.  This  suggests  the  need  for  an  optimization  technique.  The 
nature  of  the  optimization  problem  is  combinatorial,  i.e.  involves  selecting  an 
optimal  combination  of  discrete  characteristics,  and  this  type  of  optimization 
Is  best  handled  with  dynamic  programming.  The  aid  will  contain  a  dynamic 
programming  algorithm  which  "searches"  the  space  of  all  possible  combinations  of 
processing  modes.  It  would  compare  each  combination  and  select  the  better  one 
based  on  the  MAUM  described  above. 

The  aid  would  also  employ  an  information  processing  algorithm  to 
construct  the  P-j ( j )  values  needed  in  the  MAUM  functions.  This  algorithm 
assesses  the  probability  of  Identifying  a  signal  from  a  specific  type  by  using  a 
given  combination  of  processing  modes.  It  combines  pre-stored  probability  of 
Identification  values  for  specific  environmental  conditions  and  area-coverage 
factors  across  the  probabilities  of  the  various  environmental  conditions  which 
may  obtain  for  the  current  mission. 


5-6 


1 


5.1.5  Display/Entry 

The  aid  obtains  the  majority  of  the  information  it  needs  from  other 
computer  programs  and  on-board  databases,  and  produces  only  a  single  output. 

This  output,  plus  the  few  inputs  required  from  the  TACCO,  can  best  be  presented 
with  standard  alphanumeric  display/entry  techniques. 

5.1.6  Human  Judgment  Refinement/Amplification 

There  is  no  need  for  any  human  judgment  refining/amplifying  techniques 
in  this  aid.  This  situation  does  not  have  an  iterative  type  of  task  structure 
(the  same  task  repeated  over  and  over,  as  in  tracking),  so  neither  Bayesian 
Updating  nor  Adaptive  Prediction  techniques  are  appropriate.  Although  the 
problem  does  involve  optimization,  the  operator-aided  form  of  optimization  is 
not  well  suited  to  combination  problems  of  the  sort  involved  here,  because  such 
optimization  does  not  require  the  use  of  hill-climbing  (i.e.  convergence) 
algorithms. 

5.2  ATTACK  PLANNING:  AN  ATTACK-CRITERIA/WEAPQNS  PLACEMENT  DECISION  AID 

The  Attack  Planning  decision  situation  ranked  second  in  priority  for 
attack  missions,  directly  behind  the  Contact  Classification/Verification  situ¬ 
ation  which  ranked  first  for  both  types  of  missions.  Five  of  the  decision  func¬ 
tions  involved  in  attack  planning  —  Gaining  Attack  Criteria,  Determining  Air¬ 
craft  Weapon  Launch  Position,  Determining  Target  Fix,  Anticipating  Target  Move¬ 
ment,  and  Determining  Weapon  and  Setting  For  Attack  —  ranked  among  the  seven 
highest  priority  decision  functions  for  the  mission  with  attack  (see  Table  2-4). 
Interestingly,  these  same  five  decision  functions  were  the  highest  five  in  the 
ranking  for  urgency  (Table  2-6)  and  five  of  the  highest  six  in  the  ranking  for 
workload  (Table  2-7).  These  placements  characterize  these  aspects  of  attack 
planning  as  highly  time-constrained  and  work- intensive  decisions  for  the  TACCO. 


The  placement  of  these  decision  functions  in  the  MDS  (Figure  3-2)  solu¬ 
tion  space  confirms  this  characterization.  While  they  are  positioned  around  or 
near  the  middle  of  dimension  one  (information  uncertainty)  and  three  (complexity 


of  alternative  structure),  they  are  all  located  at  the  positive  extreinum  of  dimen¬ 
sion  two  (information  processing  load).  This  indicates  that  they  represent  deci¬ 
sions  which  place  a  heavy  information  load  on  the  TACCO.  These  characteristics  of 
Attack  Planning  decision  functions  explain  why  the  situation  was  assigned  such  a 
high  priority  by  the  TACCOs,  and  also  suggest  the  type  of  decision  aid  required. 


A  decision  aid  for  the  Attack  Planning  situation  should  serve  two  spe¬ 
cific  functions.  First,  it  should  automate  some  of  the  information  processing 
currently  required  of  the  TACCO,  and  second,  it  should  speed  the  rate  at  which 
phases  in  the  planning  of  an  attack  are  accomplished.  Specifically,  the  aid 
should  assist  the  TACCO  in  gaining  attack  criteria,  and  once  they  are  gained,  in 
formulating  the  optimal  tactics  for  an  attack  on  the  hostile  submarine.  Such  an 
aid  will  have  five  primary  features: 


•  automation  of  attack  criteria  -  the  aid  automatically 
interrogates  the  incoming  sensor  data  and  continuously 
compares  these  data  to  a  set  of  predefined  attack  criteria. 
When  attack  criteria  are  gained  the  aid  immediately 
notifies  the  TACCO, 

•  optimization  of  attack  tactics  -  after  attack  criteria 
are  gained,  the  aid  suggests  the  optimal  location  and 
time  for  an  attack  on  the  target  given  the  present 
location  and  motion  capabilities  of  the  submarine, 

•  optimization  of  weapon  selection  -  for  the  attack  to  be 
placed,  the  aid  suggests  the  type  of  weapon  that  should 
be  used,  and  the  optimal  engagement  setting  for  it, 

•  interface  to  pilot  and  navigator  -  if  the  TACCO  accepts 
the  attack  tactics  (i.e.  location  and  time)  suggested  by 
the  aid,  or  if  he  enters  alternate  tactics  of  his  own 
selection,  the  aid  calculates  fly-to-points  and  provides 
steering  (navigational)  commands  to  the  pilot,  and 

•  suggestion  of  tactics  for  gaining  attack  criteria  -  prior 
to  the  attainment  of  attack  criteria,  the  aid  suggests  to 
the  TACCO,  if  asked,  specific  tactics  (e.g.  deploying 
passive  sonobuoys,  deploying  active  sonobuoys,  imple¬ 
menting  MAD  search  patterns)  that  could  speed  the  pro¬ 
cess  of  gaining  attack  criteria. 


f: 
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The  Attack  Planning  aid  would  thus  be  used  to  alert  the  aircrew  when 
attack  criteria  have  been  gained,  recommend  optimum  weapon  settings,  determine 
weapon  placement,  and  provide  aircraft  navigation  commands  to  deploy  the  weapon. 

The  aid  will  be  initiated  by  the  TACCO  when  he  feels  he  is  approaching  the 
attainment  of  attack  criteria. 

There  are  many  factors  which  affect  the  way  in  which  the  aid  will  per¬ 
form  these  functions.  Acoustic  sensor  information  would  be  utilized  to  assist  the 
aid  in  determining  the  attainment  of  attack  criteria,  in  recommending  optimum  weapon 
settings,  and  in  specifying  optimal  weapon  placement.  Oceanographic  conditions 
can  have  major  impact  on  a  weapon's  ability  to  detect  and  acquire  the  target. 
Although  ASW  weapons  have  specified  acquisition  ranges,  these  ranges  are  based 
upon  a  unique  oceanographic  conditions.  Variations  in  conditions  will  impact 
upon  the  acquisition  range  of  the  weapon.  The  attainment  of  attack  criteria  can 
also  be  affected  in  other  ways  by  oceanographic  conditions,  particularly  when 
the  aircrew  is  attempting  to  prosecute  the  target  via  passive  acoustic  sensors 
and  it  is  difficult  to  determine  the  exact  distance  between  the  target  and  a 
sonobuoy. 


During  Attack  Planning,  the  target  (submarine)  will  either  be  alerted 
to  the  presence  of  the  aircraft  (and  thus  be  trying  to  avoid  the  attack),  or 
will  be  unaware  of  the  aircraft  (and  thus  be  proceeding  on  its  mission).  If  the 
aircrew  prosecutes  with  active  sonobuoy  sensors  it  can  be  assumed  the  target  has 
been  alerted  and  will  attempt  to  avoid  further  prosecution;  otherwise  it  can  be 
assumed  that  the  target  is  unaware  of  the  ASW  aircraft.  Depending  upon  the  tac¬ 
tical  situation,  time  may  be  an  important  factor  in  the  attack  planning.  If  the 
aircraft  is  running  low  on  fuel,  if  the  aircraft  is  approaching  the  end  of  its 
on-station  period,  or  if  the  submarine  is  closing  to  within  weapons  range  of  the 
friendly  force  it  is  especially  critical  to  expedite  the  attack. 
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The  attack  planning  aid  will  therefore  utilize  several  inputs  to  produce 
the  outputs  listed  above.  It  will  require  information  on: 

•  Characteristics  and  capabilities  of  the  ASW  aircraft 
and  its  weapons, 

•  Estimated  capabilities  of  the  submarine  being  attacked, 

§  Atmospheric  and  oceanographic  conditions  in  the 
region  of  the  attack, 

•  Type,  location,  depth  and  setting  of  the  deployed 
sonobuoys, 

•  History  of  the  contact  with  the  target  being  attacked,  and 

•  Overall  tactical  situation,  including  fuel  and  time 
onstation  remaining  for  the  ASW  aircraft,  the  relative 
location  of  any  friendly  areas  threatened  by  the 
submarine,  and  the  offensive  capabilities  of  the 
submarine. 

All  of  these  inputs  can  be  obtained  from  either  a  prestored  database  or  from 
other  computer  programs  in  the  ASW  aircraft.  A  brief  description  of  the  specific 
techniques  the  aid  uses  to  produce  its  outputs  from  these  inputs  is  given  below, 
in  terms  of  the  categories  in  the  decision  aiding  technique  taxonomy  (Table  5-1). 

5.2.1  Outcome  Calculator 

Some  form  of  outcome  calculator  is  needed  to  predict  the  results  of  a 
potential  attack  on  the  hostile  submarine  by  the  ASW  aircraft.  The  high  urgency 
ratings  assigned  to  the  decisions  functions  in  the  attack  planning  decision 
situation  require  the  use  of  an  highly  responsive  outcome  calculator.  The  ana¬ 
lysis  of  decision  aiding  techniques  in  the  previous  effort  (see  Zachary  1980: 
Section  3)  found  that  only  the  probabilistic,  closed- form  analytic,  or  deter¬ 
ministic  simulation  types  of  outcome  calculators  can  be  considered  as  com¬ 
putationally  fast.  The  possible  types  of  outcome  calculators  are  therefore 
restricted  to  these  three. 


A  primary  function  the  outcome  calculator  must  serve  in  the  Attack 
Planning  decision  aid  is  to  assess  the  impact  of  the  great  number  of  uncertain¬ 
ties  in  the  attack  scenario  on  its  likely  result.  There  is  uncertainty  in  the 
target's  location,  course,  speed,  and  depth,  in  the  performance  of  the  weapons 
that  can  be  used,  and  in  the  TACCO's  information  concerning  the  oceanographic 
condition.  The  outcome  calculator  must  therefore  be  able  to  accomodate  uncer¬ 
tainty  in  its  inputs,  and  incorporate  uncertainty  in  its  outputs.  Of  the  three 
types  of  calculators  listed  above,  only  probabilistic  models  meet  this  criterion. 

The  probabilistic  outcome  calculator  for  the  attack  planning  decision 
aid  should  accept  probabilistic  characterizations  of  the  target's  location, 
course,  depth,  and  speed,  capabilities  of  attack  weapons  (i.e.  the  weapon's  pro¬ 
bability  of  kill  or  P|<),  and  oceanographic  condition  data.  From  those  inputs, 
it  should  produce  a  single,  probabilistic  output  representing  the  outcome  of  the 
attack  —  a  probability  of  submarine  kill  (P^) . 

5.2.2  Value  Model 

There  are  three  criteria  that  contribute  to  the  desirability,  or  uti¬ 
lity  of  a  given  attack  plan.  The  most  important  is  the  overall  P]<  for  the 
submarine.  Also  important,  however,  are  the  time  required  (from  the  start  of 

attack  planning)  to  complete  the  attack,  and  the  impact  of  the  attack  plan  on 

the  weapons  inventory.  For  example,  if  the  ASW  aircraft  is  low  in  fuel,  then  it 
will  be  more  desirable  to  attack  sooner,  even  it  this  may  require  acting  with  a 
slightly  lower  P^.  Alternatively,  if  the  ASW  aircraft  is  low  on  weapons 
but  has  adequate  fuel  remaining,  it  will  be  more  desirable  to  wait  to  obtain  the 

highest  possible  Pr.  These  three  factors  —  Pj^ >  time,  and  weapons  inventory  — 

have  a  complex  interrelations^,  and  a  value  model  which  accurately  models  the 
necessary  tradoffs  among  them  will  necessarily  be  nonlinear.  Therefore,  the 
attack  planning  decision  aid  will  employ  a  nonlinear  value  model  to  select  among 
the  possible  attack  plans. 


5.2.3  Data  Control 

As  with  the  mode  selection  decision  aid,  this  aid  will  obtain  most  of 
its  input  information  from  other  computer  programs  in  the  aircraft's  avionics 
suite.  Since  little  or  no  input  information  will  come  directly  from  sensor  or 
operator  input  streams,  no  automatic  aggregation/disaggregation  is  necessary. 
Since  the  aid  will  not  require  a  large  database,  specialized  information  manage¬ 
ment  facilities  are  not  required.  Thus,  no  special  data  control  techniques  are 
needed  for  this  aid. 


5.2.4  Analysis 

Several  types  of  analysis  techniques  will  be  used  by  this  decision  aid. 
The  first  will  be  an  information  processing  algorithm  which  computes  the  steering 
and  navigation  commands  necessary  to  implement  the  selected  attack  plan.  This 
algorithm  incorporates  a  model  of  the  ASW  aircraft's  motion  and  maneuvering 
capabilities.  It  accepts  as  inputs  the  current  location,  course,  altitude  and 
speed  of  the  aircraft  and  the  desired  course,  altitude  and  speed  at  the  weapon 
release  point,  as  well  as  the  desired  time  of  weapon  release.  From  these,  it 
calculates  the  steering  commands  and  fly-to-points  which  will  allow  the  aircraft 
to  successfully  capture  the  weapon  release  point,  and  sends  these  to  the  naviga¬ 
tor  and  pilot. 


The  second  analysis  feature  of  the  aid  is  an  optimization  of  the  attack 
plan  —  the  calculation  of  the  optimal  weapon,  setting,  time,  and  location  for 
the  attack  on  the  submarine.  The  constraints  on  such  an  optimization  problem 
can  not  be  expressed  as  prestored  linear  equations,  because  they  will  change 
with  each  situation.  Therefore,  linear  programming  cannot  be  used.  The  nature 
of  the  optimization  is  continuous  (at  least  with  regard  to  location  of  the  attack) 
rather  than  combinatorial,  so  dynamic  programming  is  also  inappropriate.  The 
optimization  will  have  to  proceed  by  "probing"  the  space  of  possible  solutions 
and  moving  in  the  apparent  direction  of  improvement  at  each  probe.  At  each 
point  in  the  solution  space  probes,  the  probabilistic  outcome  calculator  and  the 
nonlinear  value  model  would  be  used  together  to  determine  the  effectiveness  of 
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the  attack  plan  represented  by  that  point,  and  the  slope  of  the  goodness  func¬ 
tion  in  that  'neighborhood1.  This  is  a  classic  nonlinear  programing 
formulation.  If  analysis  can  demonstrate  that  the  value  functions  will  always 
be  monomodal  over  the  space  of  all  possible  solutions,  then  the  simpler  and  more 
efficient  optimization  technique  of  Fibbonacci  search  may  be  used  instead  of 
nonlinear  programming. 

The  third  analysis  feature  of  the  aid  will  make  use  of  another  infor¬ 
mation  processing  algorithm  to  determine  automatically  the  attainment  of  attack 
criteria.  This  algorithm  will  utilize  a  prestored  set  of  attack  criteria  and 
compare  them  to  calculated  probability  areas  for  the  target.  It  will  also  con¬ 
sider  possible  effects  of  oceanographic  conditions  on  the  probability  area 
calculations.  The  algorithm  will  operate  in  real  time,  constantly  comparing  the 
latest  calculated  target  probability  area  to  the  prestored  attack  criteria. 

When  the  criteria  are  met,  the  algorithm  will  alert  the  TACCO. 

5.2.5  Display/Entry 

This  aid  will  require  both  alphanumeric  and  graphic  display  techniques. 
Graphic  displays  are  needed  to  present  the  geometry  of  the  attack  to  the  TACCO, 
pilot,  and  navigator.  They  will  also  be  needed  to  present  suggested  sensor 
deployments  which  could  speed  the  achievement  of  attack  criteria  (see  Section 
5.2.6).  Other  inputs  and  outputs  can  be  treated  adequately  with  alphanumeric 
display/entry  techniques. 

5.2.6  Human  Judgement  Refinement/Amplification 

The  outputs  provided  by  the  attack  planning  decision  aid  include  sug¬ 
gested  tactics  which  could  speed  the  attainment  of  attack  criteria.  The  tech¬ 
nique  of  Adaptive  Prediction  is  used  to  provide  this  type  of  analysis.  The 
Adaptive  Prediction  algorithm  would  record  the  various  tactics  applied  by  the 
TACCO  during  the  localization  phase  of  the  mission  and  analyze  them  according  to 
their  relative  success  or  failure.  From  these  observations  and  analysis,  it 
would  construct  a  model  of  the  tactical  approaches  that  had  highest  probability 
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of  success  for  further  localizing  the  target  at  the  current  point  in  the  mission. 
When  the  TACCO  desires  the  aid  to  suggest  tactics  for  attaining  attack  criteria, 
the  algorithm  will  use  its  adaptively  constructed  model  to  select  the  tactic  that 
Is  most  in  keeping  with  the  TACCOs  own  Inferred  tactical  preferences  and  that 
also  has  the  highest  probability  increasing  the  degree  of  target  localization. 
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CONCLUSION^  AND  RECOMMENDATIONS 


This  effort  has  produced  a  number  of  significant  results  from  both  an 
applications  and  a  methodological  prespective.  It  has  generated  a  methodology 
for  analyzing  and  prioritizing  complex  decision  spaces,  which  allows: 

•  the  identification  of  the  key  dimensions  along  which 
experienced  decision  makers  perceive  decisions  in  a  spe¬ 
cific  tactical  area, 

•  the  determination  of  the  way  in  which  other  characteristics 
of  the  decision  space,  such  as  priority,  are  constructed 
from  these  key  dimensions,  and 

•  the  construction  of  a  mathematical  function  which  assigns 
real-valued  priority  scores  to  decisions  according  to  their 
location  on  each  of  the  key  dimensions. 


The  research  has  also  produced  substantial  progress  in  the  effort  to 
apply  decision-aiding  technology  to  Naval  Air  ASM.  In  particular,  it  has 
resulted  in: 

t  The  identification  of  three  dimensions  —  degree  of  uncer¬ 
tainty,  information  processing  load,  and  complexity  of 
alternative  structure  —  which  underlie  experienced 
ASW  TACCOs  decisions, 

•  the  numerical  prioritization  of  six  Air  ASW  decision 
situations  and  the  fourteen  decision  functions  from 
which  they  were  constructed,  for  two  types  of  Air  ASW 
mission  (with  and  without  attack  on  the  submarine),  and 

•  the  construction  of  high  level  decision  aid  designs  for 
two  of  the  decision  situations  with  highest  priority. 


The  prioritization  methodology  developed  here  (Figure  1-1),  priority 
mapping ,  represents  a  new  approach  to  the  problem  of  prioritization.  The 
priority  mapping  approach  is  based  on  two  basic  assumptions  about  decision 
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making  in  general,  and  Air  ASW  decision  making  in  particular.  First,  it  assumes 
that  any  area  of  decision  making  is  a  complex  multidimensional  domain  and  that 
all  of  the  underlying  dimensions  in  this  domain  are  potentially  relevant  to  the 
priority  of  the  decisions  involved.  Second,  it  assumes  that  the  prioritization 
of  a  set  of  decisions  must  incorporate  the  intuition,  and  judgements  knowledge 
about  the  priority  of  those  decisions  held  by  individuals  experienced  in  making 
them. 

The  application  of  the  Priority  Mapping  methodology  to  the  prioritiza¬ 
tion  -of  Naval  Air  ASW  decision  situations  demonstrated  its  feasibility  and 
soundness.  The  methodology  produced  a  highly  useful  dimensional ization  of  the 
ASW  decision  space  (Figure  3-1)  as  well  as  mission-sensitive  prioritization  of 
the  six  decisions  situations  identified  in  the  previous  effort  (Tables  4-8  and 
4-9). 

The  successful  prioritization  of  the  decision  situations  allowed  the 
Initial  steps  to  be  taken  in  the  design  of  decision  aids  for  the  highest  priority 
situations.  An  Optimal  Mode-Selection  decision  aid  was  outlined  for  the  Contact 
Classification/Verification  situation  (which  achieved  the  highest  priority  for 
both  types  of  ASW  mission).  This  aid  uses  a  dynamic  programming  algorithm  and  a 
multi  attribute  utility  model  together  with  computational  algorithms  for  deter¬ 
mining  the  probability  of  correctly  identifying  a  signal  from  a  particular 
target  given  a  specific  signal -processing  modes  and  environmental  conditions  to 
help  the  TACCO  or  SENSO  select  an  optimal  mode  of  processing  for  his  acoustical 
signal  processing  equipment  (Section  5.1).  A  decision  aid  was  outlined  for  the 
Attack  Planning  decision  situation,  which  achieved  the  second  highest  priority 
for  missions  with  attack.  This  aid  uses  a  real-time  information  processing 
algorithm  to  automate  the  determination  of  attack  criteria  attainment.  It  sub¬ 
sequently  uses  a  probabilistic  model  outcome  calculator,  a  nonlinear  value 
model,  and  a  nonlinear  programming  algorithm  to  determine  the  tactics  for  the 
optimal  attack  that  can  be  placed  on  the  hostile  submarine  (Section  5.2). 


This  effort  has  developed  and  applied  a  new  methodology  for  determining 
which  decisions  in  a  specific  context  should  have  priority  for  decisions  aiding. 
It  has  also  begun  the  design  of  decision  aids  for  the  two  most  critical  decision 
situations  by  identifying  the  specific  techniques  that  are  most  applicable  to 
those  situations.  But  even  though  the  suggested  mix  of  techniques  may  be  the 
best  possible  for  each  situation,  it  is  not  certain  that  even  their  optimal 
application  can  materially  increase  the  quality  of  the  decisions  made.  The 
degree  of  uncertainty  involved  in  the  problems,  the  required  response  times,  or 
other  factors  could  impose  such  severe  constraints  that  in  practice,  these  deci¬ 
sion  problems  can  not  be  substantially  aided.  Further  investigation  is  needed 
to  demonstrate  that  the  highest  priority  decisions  are  in  fact  amenable  to 
aiding  by  the  selected  decision  aiding  approach. 

The  discussions  with  the  experienced  TACCOs  at  Moffett  Field,  however, 
indicated  that  additional  consideration  should  be  given  to  the  potential  impact 
of  the  overall  aid  design  on  the  human  operator.  Many  TACCOs  expressed  a  view 
that  "they  were  working  for  the  computer,  rather  than  the  other  way  around," 
and  were  concerned  that  the  introduction  of  additional  automation  into  their 
workstation  might  further  aggravate  this  situation.  It  is  clear,  though  often 
unrecognized,  that  decision  aids  do  not  "by  definition"  reduce  the  operator's 
workload.  An  aid  introduces  new  tasks  that  must  be  performed,  and  if  in  doing 
so  it  does  not  obviate  others,  the  net  effect  can  be  a  worsening  of  an  already 
difficult  situation  for  the  operator. 

The  potential  improvement  in  decision  performance  that  may  result  from 
a  candidate  decision  aid  must  be  considered  in  conjunction  with  the  importance 
of  the  decision  being  aided  in  determining  whether  the  full  development  of  that 
decision  aid  is  truly  warranted.  It  is  therefore  recommended  that  future 
efforts  also  be  directed  towards  determining  the  possible  inputs  that  these  can¬ 
didate  and  designs  could  have  on  the  operator  in  those  high  priority  decision 
situations. 


APPENDIX  A 

TEXT  OF  INTRODUCTORY  BRIEFING  GIVEN  TO  P-3C  TACCOS 
PRIOR  TO  MULTIDIMENSIONAL  SCALING  INTERVIEWS 


A.  TEXT  OF  INTRODUCTORY  BRIEFING  GIVEN  TO  P-3C  TACCOS 
PRIOR  TO  MULTIDIMENSIONAL  SCALING  INTERVIEWS 


I  would  like  to  begin  by  thanking  you  all  for  coming  here  today  to  help 
us  In  this  effort.  You've  been  asked  to  come  here  because  of  your  familiarity 
with  ASW  and  ASW  decision  making.  Before  we  ask  you  some  specific  questions 
about  ASW,  I'd  like  to  begin  by  providing  a  brief  background  on  what  it  is  that 
we  mean  by  decision  aids  and  decision  aiding,  what  the  overall  structure  of  our 
program  is,  where  we  have  been,  and  why  we  have  come  here  to  talk  to  you.  As 
good  a  place  as  any  to  begin  is  with  the  definition  of  a  decision  aid. 

Very  simply  put,  a  decision  aid  is  any  kind  of  device  that  helps  humans 
make  better,  more  efficient,  clearer,  and  faster  decisions.  Now,  obviously,  a 
wide  range  of  possible  things  can  be  considered  decision  aids  —  from  a  pencil 
and  a  sheet  of  paper  which  enable  you  to  do  calculations  to  large  computer 
systems  and  programs.  What  we're  most  interested  in  are  specific  tools  that 
will  enable  you  as  a  TACCO  to  interact  with  your  on-board  computer  system  to 
help  you  make  the  kinds  of  decisions  that  you  have  to  make  in  the  course  of  an 
ASW  mission. 

The  Navy's  interest  in  decision  aiding  has  been  increasing  significantly 
recently  because  of  the  realization  that  warfare,  in  general,  and  ASW,  in  partic¬ 
ular,  is  becoming  more  automated  and  more  highly  technological.  The  speed  and 
complexity  with  which  decisions  must  be  made  is  increasing  constantly  to  a  point 
where,  in  the  not  too  distant  future,  you  as  TACCOs,  will  be  overloaded,  possibly 
beyond  your  capability  to  make  the  necessary  decisions  within  a  reasonable  amount 
of  time.  Therefore,  the  Navy  is  interested  in  developing  computerized  systems 
of  decision  aids  that  will  help  you  keep  pace  with  the  increasing  automation  on¬ 
board  your  aircraft.  It  should  be  pointed  out  that  these  decision  aids  will  not 
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take  you  out  of  the  decision  making  process  or  automate  your  functions.  Rather, 
they  will  provide  you  with  better,  more  intelligent  support  from  machines  and 
will  give  you  time  to  do  what  you  do  best  —  think  and  make  decision.  The  whole 
concept  of  decision  aiding  is  based  on  the  notion  that  the  most  complex,  useful, 
and  important  piece  of  equipment  on  any  platform  is  the  human  brain.  Humans  are 
onboard  to  make  decisions;  but  computers  and  other  kinds  of  devices  can  assist 
by  managing  information,  making  certain  kinds  of  data  available  at  your  finger¬ 
tips,  helping  you  remember  things  (like  the  procedures  you  must  go  through  to 
accomplish  a  specific  function),  performing  certain  kinds  of  calculations  for 
you,  and  so  forth. 

There  are  some  simple  examples  of  decision  aids  currently  on-board  ASW 
platforms  such  as  the  P-3C  and  S-3A.  For  example,  the  tactics  pattern  matrix 
feature  of  the  14.4  program  on  the  P-3C  supports  you,  as  the  TACCO,  in  your 
selection  and  deployment  of  patterns  of  sonobuoys  through  a  series  of  cues.  The 
basic  pattern  is  then  constructed  automatically  by  the  computer.  Another 
example  is  the  project  position  function  which  automatically  establishes  and 
displays  a  past  or  future  projection  of  a  target  position. 

Some  high  level  decision  aiding  is  also  available  in  the  ESM  gear.  You, 
as  a  TACCO,  can,  for  example,  select  up  to  20  signals  to  be  automatically 
displayed.  Only  the  pre-selected  signals  will  then  be  reported  to  you  if  they 
are  identified  by  the  automatic  signal  processing  equipment.  The  PROTEUS  system, 
which  is  soon  to  be  incorporated  to  the  P-3C,  contains  a  number  of  decision  aids 
of  this  same  variety  as  does  the  PAD/C  system  for  passive  acoustical  signal 
processing. 

Decision  aids  can  work  at  a  variety  of  levels.  Simple  decision  aids 
can,  for  example,  provide  very  rudimentary  bookkeeping  functions  or  provide 
checklists  of  things  that  must  be  done.  More  complex  decision  aids  can  "ask" 
you  the  questions  that  you  should  be  asking  yourself.  Still  more  complex  deci¬ 
sion  aids  can  anticipate  some  of  the  information  that  you  may  want,  based  on 
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your  past  performance;  they  can  perform  certain  kinds  of  calculations  automati¬ 
cally  so  that  the  results  of  the  calculations  will  be  available  when  requested; 
they  can  make  certain  kinds  of  inferences  about  what  might  happen  in  the  future 
enabling  you  to  play  "what  if"  games  to  find  out  what  results  might  be  obtained 
if  a  specific  course  of  action  is  undertaken.  These  latter  things,  which  are  at 
the  higher  end  of  the  spectrum  of  the  capabilities  of  decision  aids,  are  the 
ones  that  we're  most  interested  in. 

To  give  you  a  feeling  for  the  types  of  aids  that  are  possible.  I'd  like 
to  review  some  of  the  other  decision  aid  projects  of  this  type  that  are  on  going 
today  in  the  Navy.  The  largest  decision  aiding  effort  to  date  has  been  under¬ 
taken  by  the  Office  of  Naval  Research.  Their  program  has  concentrated  on  devel¬ 
oping  a  variety  of  decision  aids  for  carrier-based  air  strike  operations.  Some 
of  the  specific  problems  to  be  addressed  by  these  decision  aids  are:  planning 
the  ingress  route  for  an  incoming  carrier-based  air  strike  or  reconnaissance 
mission  through  a  complex  sensor  field;  determining  the  specific  timing  of  alpha 
type  air  strikes;  task  force  EMCOM  planning;  and  overall  air  strike  campaign 
planning.  The  decision  aids  being  developed  by  ONR  are  very  high  level  decision 
aids,  both  in  the  sense  that  they  provide  a  great  deal  of  assistance  to  the 
human  and  in  the  sense  that  they  deal  with  very  high  level  command  and  control 
decisions  (at  the  task  force  commander  level  or  higher). 

More  germaine  to  our  discussion  today  is  the  effort  that  we  have  under¬ 
taken  under  the  joint  sponsorship  of  the  Naval  Air  Development  Center  and  the 
Office  of  Naval  Research  to  identify  possible  decision  aids  for  on-platform  ASW 
operations.  I'd  like  to  review  some  of  the  earlier  parts  of  this  effort  to 
clarify  why  we  are  here  today. 

We  began  our  effort  by  looking  at  the  ASW  platforms  that  will  be  in  use 
in  the  1980-1985  timeframe  —  the  P-3C,  the  S-3A,  and  the  LAMPS  MK-III.  We 
examined  the  specific  missions  that  are  undertaken  by  these  three  platforms  in 
order  to  define  a  generic  or  generalized  ASW  mission,  to  identify  commonalities 
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in  the  missions  flown  by  the  three  platforms,  and  to  identify  some  fo  the  criti¬ 
cal  differences  both  in  crew  functions  and  in  the  details  of  the  missions  that 
were  undertaken.  We  then  constructed  this  flowchart*  of  the  sequence  of  opera¬ 
tions  that  takes  place  in  the  generalized  mission.  The  mission  was  subdivided 
into  very  broad  categories  —  movement  to  the  search  area,  on-station  search, 
prosecution  of  the  contract,  possibly  culminating  in  attack  and  destruction  of  a 
hostile  submarine.  These  stages  were  then  subdivided  into  the  detailed  steps 
that  involved  specific  decisions  about  classification,  and  extension  of  search 
area,  and  anticipation  of  target  movement.  We  identified  the  specific  sequences 
in  which  these  decisions  were  made,  recognizing  that  the  interrelations  between 
these  decisions  are  highly  dependent  on  the  sequential  nature  of  the  ASW 
mission.  The  decisions  that  take  place  in  the  attack  phase,  for  example,  are 
dependent  upon  the  successful  completion  of  those  portions  of  the  mission  that 
relate  to  search  and  early  prosecution  of  a  contact.  Ultimately,  we  identified 
six  broad  areas  that  we  termed  decision  making  situations.  We  defined  a  deci¬ 
sion  making  situation  as  a  portion  of  a  mission  in  which  a  complex  sequence  of 
decisions  has  to  be  made  by  the  TACCO.  These  situations  were  complex  because 
they  involved  trading  off  a  number  of  factors  against  one  another.  The  six 
decision  situations  were:** 

(1)  On-Station  Search, 

(2)  Contact  Classification/Verification, 

(3)  Target  Localization, 

(4)  Surveillance  Tracking, 

(5)  Lost  Contact  Reacquisition,  and 

(6)  Attacking  Planning. 


♦Figure  A-l  displayed  at  this  point. 

♦♦Figure  A-l  removed,  and  Figure  A-2  displayed  at  this  point. 
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ASW  PROBLEM-SOLVING  DECISION  SITUATIONS 


•  ON-STATION  SEARCH 

•  CONTACT/CLASSIFICATION  VERIFICATION 

•  CONTACT  LOCALIZATION 

•  LOST  CONTACT  REACQUISITION 

•  SURVEILLANCE  TRACK 

•  ATTACK  PLANNING 

Figure  A-2.  "ASW  Problem  Solving  Decision  Situations"  Vugraph 


The  next  Issue  we  addressed  in  our  program  was  the  relative  priority  of 
each  of  these  situations.  By  prioritizing  the  decision  situations,  we  felt  that 
we  could  determine  which  situation  could  most  benefit  from  decision  aiding.  But 
we  were  Immediately  confronted  by  the  problem  that  there  was  no  single  dimension, 
or  criteria,  by  which  we  could  prioritize  the  decision  situations.  They  obvi¬ 
ously  are  Influenced  by  their  sequence  in  the  mission.  They  are  also  Influenced 
by  varying  time  constraints  on  the  decisions  that  must  be  made  in  each  situation 
and  how  busy  the  TACCO  is  in  each  situation.  The  more  we  thought  about  it,  the 
more  different  criteria  for  prioritization  we  were  able  to  define.  It  became 
clear  to  us  that  one  of  the  problems  was  that  we,  as  analysts,  could  not  deter¬ 
mine  the  prioritization.  We  decided  that  the  only  way  to  determine  the  specific 
criteria  that  were  relevant  to  the  prioritization  of  these  decisions  was  to  ask 
the  people  who  made  these  kinds  of  decisions,  people  such  as  yourselves.  That 
is  why  we  are  here  today.  We  want  to  determine  how  the  various  decisions  that 
you,  as  TACCOs  make,  should  be  prioritized.  To  do  this,  we  have  to  determine 
the  dimensions  or  criteria  by  which  these  decisions  are  interrelated  in  your 
minds.  Then  we  have  to  determine  the  importance  or  priority  of  these  decisions 
In  a  mission.  * 

There  are  a  number  of  techniques  that  can  be  used  to  do  this.  One  way 
would  be  to  ask  you,  in  very  lengthy  and  detailed  discussions,  to  try  to  iden¬ 
tify  the  dimensions  which  are  salient  to  ASW  decision  making  for  you.  But  be¬ 
sides  taking  a  lot  of  your  time.  It  Is  not  clear  that  that  technique  would  work. 
People  are  often  very  unclear  about  the  underlying  principles  they  use  to  think 
about  common,  everyday  things,  like  decision  making.  In  addition,  we  would  have 
the  problem  of  resolving  the  differences  we  encountered  between  the  various 
people  we  talked  to.  So  Instead,  we  have  decided  to  use  a  more  formal  mathema¬ 
tical  technique  known  as  multidimensional  scaling  which  will  take  less  of  your 
time  and  will  enable  us  to  determine  both  the  dimensions  and  the  relative  impor¬ 
tance  of  the  various  decisions  from  the  same  set  of  data.  Multidimensional 


scaling  uses  the  computer  program  to  calculate  the  dimensions  from  very  simple 
judgements  made  by  you  about  the  basic  similarity  or  difference  among  these 


various  decisions.  Me  also  decided  that  we  wanted  to  address  not  just  the  broad 
analytic  categories  that  we  have  called  decision  situations,  but  more  precise, 
meaningful,  specific  decisions  that  are  made  by  TACCOs.  We  identified  14  of 
these  decisions,  many  of  which  occur  in  several  of  the  decision  situations. 

There  are,  of  course,  many  more  decisions  that  are  made  in  the  course  of  a 
mission  but  the  14  decisions  that  we  chose  were  ones  that  appeared  in  more  than 
a  single  decision  situation  or  ones  that  seemed  particularly  amenable  to  decision 
aiding.  The  14  decisions  that  we  have  chosen  are  shown  in  this  vugraph.* 

You  have  in  front  of  you  a  set  of  cards.**  Each  card  describes  one  of 
these  decisions.  We're  going  to  ask  you  to  make  judgments  about  which  decisions 
you  feel  are  similar  or  dissimilar,  and  to  rank  them  by  various  criteria.  The 
results  of  these  judged  similarities  and  rankings  will  be  used  by  the  multidi¬ 
mensional  scaling  process  to  mathematically  determine  a  set  of  relationships 
between  these  decisions  that  will  help  us  in  prioritizing  them.  It  will  also 
help  us  to  understand  the  kinds  of  distinctions  that  you  find  most  relevant 
among  these  various  decisions.  But,  most  importantly,  we  feel  it  will  enable  us 
to  relate  our  analysis  in  a  concrete  way  to  your  knowledge,  experience,  and 
intuition  of  the  ASM  mission.  With  your  help,  we  will  be  able  to  determine  the 
best  places  to  apply  decision  aiding  techniques  in  the  ASW  mission. 

I'd  like  to  add  one  other  note  about  the  way  in  which  you  should  sort 
and  rank  these  decision  functions  in  later  portions  of  this  procedure.  Do  not 
sort  or  rank  the  decisions  in  terms  of  how  you  think  a  decision  aid  could  help 
you  make  these  decision  nor  in  terms  of  how  you  think  they  could  be  made  better, 
but  rather  in  terms  of  how  you  currently  go  about  making  these  decisions  and  how 
they  currently  are  handled  on  the  platforms  on  which  you  have  worked. 


*Figure  A-2  removed  at  this  point,  and  Figure  A-3  displayed  and  left  on 
screen  throughout  the  remainder  of  the  procedures. 

**Card-packs  describing  the  14  decision  functions  passed  out  to  inter¬ 
viewees  at  this  point. 
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DECISION  FUNCTIONS  PERFORMED  BY  THE  TACCO 


•  CREATE  SENSOR  PATTERN 

•  CONSTRUCT  SENSOR  MONITORING  PATTERN 

•  COMPENSATE  FOR  ACOUSTIC/ATMOSPHERIC  PROPAGATION  CONDITIONS 

•  ADJUST  PATTERN  TO  SENSOR  FAILURE 

•  DETERMINE  SIGNAL  IS  A  VALID  CONTACT 

•  CLASSIFY  SIGNAL 

•  MANAGE  EQUIPMENT/STORES  TO  ACCOMMODATE  PRESENT 
AND  FUTURE  NEEDS 

•  EXTEND  SENSOR  PATTERN 

•  ANTICIPATE  TARGET  MOVEMENT 

•  DETERMINE  TARGET  FIX 

•  GAIN  ATTACK  CRITERIA 

•  DETERMINE  AIRCRAFT  ATTACK  LAUNCH  POSITION 

•  DETERMINE  WEAPON  AND  SETTING  FOR  ATTACK 

•  COORDINATE  HAND-OFF 


Figure  A-3.  "Decision  Functions  Performed  by  the  TACCO"  Vugraph 
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B.  RESPONSE  FORMS  FOR  MULTIDIMENSIONAL  SCALING  INTERVIEWS 


This  appendix  contains  the  forms  on  which  the  TACCO's  Interviewed  at 
Moffett  Field  recorded  the  results  of  the  unconstrained  sorting  and  four 
rankings.  All  of  the  pages  in  the  appendix  were  given  to  each  interviewee  as  a 
stapled  packet.  The  first  page  was  used  to  record  some  general  biographical 
information  on  the  respondent.  This  information  was  used  in  grouping  the 
unconstrained  sorting  responses  for  subgroup  comparisons  (see  Section  3.3). 

The  next  two  pages  provided  the  instructions  and  response  sheet  for  the 
unconstrained  sorting  of  the  ASW  decisions.  The  remaining  pages  provided  the 
instructions  and  response  forms  for  four  different  rankings  of  the  fourteen 
decisions. 
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BIOGRAPHICAL  INFORMATION 


Aircraft  Type:  _ 

Rank/Designator:  _ 

Organization:  _ 

Date  TACCO  Designation:  _ 

Date  Mission  Commander  (If  applicable):  _ 

Deployment  Locations  as  TACCO  (and  dates): 
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INSTRUCTIONS  FOR  THE  ‘SORTING*  TASK 


You  have  been  given  a  pack  of  fourteen  yellow  cards.  On  each  of  these 
cards  is  a  decision  or  problem  that  is  encountered  in  an  ASW  mission.  You  have 
all  probably  faced  these  problems  many  times  in  your  experience  as  TACCOs.  Each 
decision  is  in  some  way  different  from  all  the  others,  but  each  decision  is  not 
totally  unique;  some  of  the  decisions  are  more  alike  than  others.  What  we  would 
like  you  to  do  is  arrange  these  decisions  and  problems  into  groups  according  to 
how  similar  they  are.  That  is,  if  there  are  a  number  of  cards  which  represent 
decisions  or  problems  you  feel  that,  based  on  your  experience,  are  similar,  then 
place  all  these  cards  together.  If  there  is  a  card  which  you  think  is  suf¬ 
ficiently  unique  that  it  isn't  similar  to  any  of  the  others,  then  place  it  in  a 
group  by  itself.  There  is  no  limit  on  the  number  of  groups  you  can  make  or  on 
the  number  of  cards  you  can  place  in  each  group.  While  the  final  definition  of 
what  constitutes  similar  decisions  is  left  to  you,  we  would  like  you  to  think  of 
it  as  referring  to  decisions  or  problems  which  you  somehow  solve  in  the  same  way 
or  which  place  similar  demands  on  you  as  a  TACCO. 
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INSTRUCTIONS :  Record  each  group  of  decisions  you  have  formed  on  a  separate 
block  below.  Take  one  group  and  write  the  code  for  the  decisions  in  it  on  the 
blank  line  provided.  Then  do  the  same  for  another  group  in  the  next  block  until 
each  group  has  been  recorded  in  a  separate  block.  If  you  wish,  you  may  also 
include  a  short  phrase  describing  the  similarity  you  saw  in  that  group  of 
decisions. 


GROUP 


GROUP 


GROUP 


GROUP 


GROUP 


GROUP 


GROUP 


INSTRUCTIONS  FOR  RANKING  1 


We  would  now  like  you  to  rank  these  decision  problems  In  the  order  In 
which  a  less-than-optimal  decision  would  have  the  most  detrimental  effect  on  the 
mission.  The  decision  problem  for  which  a  less-than-optimal  decision  would  have 
the  least  detrimental  impact  on  the  mission  should  be  ranked  last,  and  the  deci¬ 
sion  problem  for  which  a  less-than-optimal  decision  would  have  the  greatest 
detrimental  impact  on  the  mission  should  be  ranked  first.  For  the  mission, 
assume  that  it  is  an  ASM  mission  in  which  the  submarine  is  to  be  attacked  and 
destroyed  if  possible. 
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INSTRUCTIONS.  Enter  the  two- letter  codes  for  the  decisions  below  in  the  order 
in'whfcR  you  ranked  them.  That  is,  write  the  code  for  the  decision  you  ranked 
first  next  to  for  the  decision  you  ranked  second  next  to  '2.'  and  so  on. 


1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 

11. 

12. 

13. 

14. 
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INSTRUCTIONS  FOR  RANKING  2 


Rank  the  decisions  according  to  the  same  criterion  as  In  the  previous 
case,  but  for  the  mission,  assume  an  ASW  mission  In  which  the  submarine  Is  only 
to  be  tracked  and  handed-off  to  a  relief  platform. 


rm 


B-7 


INSTRUCTIONS.  Enter  the  two- letter  codes  for  the  decisions  below  in  the  order 
in  which  you  ranked  them.  That  Is,  write  the  code  for  the  decision  you  ranked 
first  next  to  *1.',  for  the  decision  you  ranked  second  next  to  *2.'  and  so  on. 


1. 

2. 

3. 


5. 

6. 

7. 

8. 

9. 

10. 
11. 
12. 

13. 

14. 
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INSTRUCTIONS  FOR  RANKING  3 


Rank  the  decisions  according  to  their  urgency.  Define  urgency  as 
referring  to  the  speed  with  which  the  decision  has  to  be  made  once  you  know  it 
must  be  made. 


0 


INSTRUCTIONS.  Enter  the  two-letter  codes  for  the  decisions  below  in  the  order 
in  which  you  ranked  them.  That  is,  write  the  code  for  the  decision  you  ranked 
first  next  to  for  the  decision  you  ranked  second  next  to  *2.'  and  so  on. 


1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 

11. 

12. 

13. 

14. 


INSTRUCTION  FOR  RANKING  4 


Rank  these  decisions  according  to  your  workload  during  each  of  them. 
Consider  both  your  cognitive,  or  mental  workload  and  your  physical  workload  in 
ranking  the  decisions.  Rank  the  decision  during  which  your  workload  is  heaviest 
first,  and  the  one  during  which  your  workload  is  least  heavy  last. 
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INSTRUCTIONS.  Enter  the  two-letter  codes  for  the  decisions  below  in  the  order 
in  which  you  ranked  them.  That  is,  write  the  code  for  the  decision  you  ranked 
first  next  to  for  the  decision  you  ranked  second  next  to  '2.'  and  so  on. 


1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 

11. 

12. 

13. 

14. 
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APPENDIX  C 

RESULTS  OF  UNCONSTRAINED  SORTINGS  OF  ASW  DECISIONS 


C.  RESULTS  OF  UNCONSTRAINED  SORTINGS  OF  ASW  DECISIONS 


This  appendix  presents  the  results  of  unconstrained  sortings  of  four¬ 
teen  air  ASW  decisions  by  32  P-3C  Tactical  Coordinators  (TACCOs)  stationed  at 
NAS,  Moffett  Field.  The  sortings  were  performed  as  part  of  interviews  conducted 
at  Moffett  Field  between  10  December  1979  and  14  December  1979.  The  full  inter¬ 
view  procedure  is  described  in  Section  2  of  this  report. 

The  fourteen  decisions  that  were  sorted  are  shown  in  Table  C-l.  The 
instructions  for  the  sorting  task,  and  a  sample  of  the  form  on  which  the  results 
were  recorded,  are  given  in  Appendix  B.  Because  one  of  the  interviewees 
apparently  misunderstood  the  instructions  and  gererated  an  unusable  sorting, 
results  for  only  31  sortings  are  presented  here. 

Each  sorting  is  presented  as  several  lists  of  decisions.  All  the  deci¬ 
sions  on  each  list  were  placed  into  a  group  by  the  indicated  individual.  In  the 
lists,  each  decision  function  is  identified  only  by  a  two-letter  code.  The  code 
used  for  each  decision  is  also  given  in  Table  C-l.  The  order  in  which  the  indi¬ 
viduals  are  listed  below  is  arbitrary,  but  it  corresponds  to  the  order  in  which 
they  are  listed  in  Appendix  D,  where  the  ranking  results  are  presented. 
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Tabic  C-1.  Fourtaan  ASW  Dads  on  Functions  and  Alphabetic  Codes 


CODE 

DECISION  FUNCTION 

AP 

ADJUST  PATTERN  TO  SENSOR  FAILURE 

EP 

EXTEND  SENSOR  PATTERN 

AM 

ANTICIPATE  TARGET  MOVEMENT 

MP 

CONSTRUCT  SENSOR  MONITORING  PATTERN 

CH 

COORDINATE  HAND-OFF 

VC 

DETERMINE  SIGNAL  IS  A  VALID  CONTACT 

DS 

DETERMINE  WEAPON  AND  SETTING  FOR  ATTACK 

AC 

GAIN  ATTACK  CRITERIA 

TF 

DETERMINE  TARGET  FIX 

CP 

CREATE  SENSOR  PATTERN 

ME 

MANAGE  EQUIPMENT/STORES  TO  ACCOMODATE  PRESENT  AND  FUTURE  NEEDS 

CS 

CLASSIFY  SIGNAL 

PC 

COMPENSATE  FOR  IN  SITU  ACOUSTICAL 

AND  ATMOSPHERIC  PROPAGATION  CONDITIONS 

LP 

DETERMINE  AIRCRAFT  LAUNCH  POSITION  FOR  ATTACK  ON  TARGET 

INDIVIDUAL  1 


GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  PC,CP,MP,AP,ME 

2  CS.VC 

3  AM,EP,TF 

4  AC,D$,LP 

5  CH 


INDIVIDUAL  2 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  CS.VC 

2  PC 

3  ME.MP 

4  AP.CP.EP 

5  TF,AC,LP,AM,DS 

6  CH 


INDIVIDUAL  3 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  DP, ME, AM, MP 

2  VC.CS 

3  AC,LP,DS,PC,EP,AP,TF 

4  CH 


INDIVIDUAL  4 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  CP,EP,AP 

2  CH 

3  ME 

4  VC, PC 

5  CS 

6  MP.LP 

7  AC,AM,DS,TF 
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INDIVIDUAL  5 


GROUP  DECISION  FUNCTIONS  IN  GROUP 


1 

2 

3 

4 

5 

6 
7 


AP,CP,PC,EP,MP 

CH 

TF.AM.AC 

CS.VC 

ME 

OS 

LP 


INDIVIDUAL  6 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  ME  ,CP  ,MP .PC , AP , EP 

2  CS  VC 

3  tf’am.ac.ds.lp.ch 


INDIVIDUAL  7 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  TF.AM.AC.DS.LP 

2  CS.VC 

3  PC.CP.EP.AP.MP 

4  ME 

5  CH 


INDIVIDUAL  8 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  CP,PC,MP,AP,EP 

2  VC.CS 

3  TF.AM.AC 

4  DS.LP 

5  CH 
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INDIVIDUAL  9 


GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  vc.cs 

2  MP.CP 

3  AP.EP.PC 

4  AC,AM,LP,TF,DS 

5  ME.CH 


INDIVIDUAL  10 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  CS.VC 

2  AP,EP,MP,PC,CP 

3  ME 

4  CH 

5  DS,LP,AM,TF,AC 


INDIVIDUAL  11 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  CP 

2  PC.EP.ME 

3  AP.MP 

4  CS.VC, TF, AM 

5  AC.DS.LP 

6  CH 


INDIVIDUAL  12 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  CP.EP 

2  MP,PC,ME,AP 

3  CS.VC 

4  TF.AM 

5  AC.DS.LP 

6  CH 
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INDIVIDUAL  13 


GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  CP, PC, MP, ME 

2  AP , EP ,  AM 

3  AC,DS,LP 

4  VC,CS,TF 

5  CH 


INDIVIDUAL  14 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  ME 

2  VC.CS.CH 

3  PC,CP,EP,AP,MP 

4  TF,AM,AC,LP,DS 


INDIVIDUAL  15 

GROUP  DECISION  FUNCTIONS  IN  GROUP- 

1  CH 

2  cs  vr 

3  AM*TF,DS,LP,AC 

4  CP ,MP , EP , ME , AP , PC 


INDIVIDUAL  16 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  CP.MP,  EP,AP,PC 

2  ME 

3  CS.VC 

4  TF.AM,  AC 

5  LP,DS 

6  CH 


INDIVIDUAL  17 


GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  VC.CS 

2  TF,AM 

3  CP,MP,EP,AP,PC 

4  CH 

5  DS,AC,LP 

6  ME 


INDIVIDUAL  18 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  TF,AM,LP,AC 

2  CP  ,ME  ,PC  ,MP  ,EP ,  AP 

3  CS.VC 

4  DS 

5  CH 


INDIVIDUAL  19 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  EP,MP,PC,CP,ME,AP 

2  VC  CS 

3  tf’am.ac.ds.lp 

4  CH 


INDIVIDUAL  20 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  PC,MP,CP,AP,EP 

2  ME 

3  VC,C$ 

4  AM,TF 

5  LP,AC,OS 

6  CH 
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INDIVIDUAL  21 


GROUP  DECISION  FUNCTIONS  IN  GROUP 


_ 

1 

ME, PC, CP 

2 

AP,MP,EP 

•  - 

3 

VC,CS 

4 

AM,TF,AC 

5 

DS,LP 

-  . 

6 

CH 

INDIVIDUAL 

22 

... 

GROUP 

DECISION  FUNCTIONS  IN  GROUP 

-i 

1 

1 

AM.EP 

k 

2 

TF 

I 

3 

ME.CH 

l 

4 

VC.CS 

5 

PC.DP.DS 

7 

6 

AC 

7 

LP 

8 

AP.MP 

-• 

|i  INDIVIDUAL  23 


GROUP  DECISION  FUNCTIONS  IN  GROUP 


\ 

1 

TF,AC,AM,LP,DS 

] 

2 

MP,ME,AP 

3 

CP.EP 

:| 

4 

CS.VC.PC 

1 

5 

CH 

t 

INDIVIDUAL 

24 

'  1 

1 

.1 

GROUP 

DECISION  FUNCTIONS  IN  GROUP 

1  LP.DS 

2  CH 

3  AM,TF,AC 

4  EP,CP ,PC 

5  VC.CS 


INDIVIDUAL  25 


GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  CH 

2  VC.CS 

3  TF.AC 

4  AM.DS.LP 

5  AP,EP,M£,PC,MP ,CP 


INDIVIDUAL  26 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  PC,CP,MP,EP,AM,TF,AC 

2  ME.AP.DS.CH 

3  CS.VC 

4  LP 


INDIVIDUAL  27 

GROUP  DECISION  FUNCTIONS  IN  GROUP 

1  CP,EP,MP,PC,AP 

2  ME 

3  CS.VC 

4  AM 

5  TF 

6  CH 

7  AC.DS.LP 


INDIVIDUAL  28 

GROUP  DECISION  FUNCTIONS  IN  GROUP 


j  INDIVIDUAL 

1  GROUP 

29 

-DECISION  FUNCTIONS  IN  GROUP 

i  1 

PC,CP,AP,MP,£P 

2 

CS,VC,TF 

1  3 

ME 

!  4 

LP,DS,AC,AM 

5 

CH 

INDIVIDUAL 

GROUP 

30 

DECISION  FUNCTIONS  IN  GROUP 

1 

ME,CP,PC,AP,MP,EP 

2 

YC.CS 

3 

AM,TF,AC 

4 

DS.LP 

5 

CH 

j  INDIVIDUAL 

31 

GROUP 

DECISION  FUNCTIONS  IN  GROUP" 

CP,PC,MP,ME,EP,AP 

i  2 

CS.VC 

1  3 

CH 

4 

LP,AM,TF,AC 

1  5 

DS 

■j 
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0.  RESULTS  OF  RANKINGS  OF  ASW  DECISIONS 


This  appendix  presents  the  results  of  rankings  of  fourteen  Air  ASW 
decisions  (listed  in  Table  C-l)  by  32  TACCOs  from  NAS  Moffett  Field.  These 
rankings  were  performed  as  part  of  the  interview  described  in  Section  2.  The 
fourteen  decisions  were  ranked  according  to  four  different  criteria.  The 
instructions  for  these  rankings  and  the  forms  on  which  the  results  were  recorded 
are  given  in  Appendix  B.  The  first  ranking  was  done  according  to  the  perceived 
importance  of  the  decision  in  a  mission  where  the  objective  is  to  attack  and 
destroy  the  hostile  submarine.  The  second  ranking  was  done  according  to  the  per¬ 
ceived  importance  of  the  decision  function  in  a  mission  where  the  objective  is 
to  survey  the  submarine  only.  The  third  ranking  was  done  according  to  the  pre- 
ceived  urgency  of  the  decisions  in  whatever  type  of  mission  gave  them  the 
greatest  urgency.  The  fourth  ranking  was  done  according  to  the  TACCO's  per¬ 
ceived  workload  (both  cognitive  and  physical)  during  each  of  the  fourteen  deci¬ 
sion  functions. 

The  results  of  these  four  rankings  are  given  below.  The  ranking 
generated  by  each  individual  using  each  criteria  is  presented  as  a  list  of  the 
two-letter  codes  used  in  the  interviews  to  represent  the  decision  functions. 

Table  C-I  (in  Appendix  C)  contains  the  full  decision  name  represented  by  each 
two-letter  code.  The  order  in  which  the  individuals  are  listed  is  arbitrary, 
but  is  the  same  for  all  four  rankings,  and  corresponds  to  the  order  in  which  the 
unconstrained  sortings  are  presented  in  Appendix  C.  The  single  individual 
omitted  in  Appendix  C  is  individual  number  eight  here.  Thus,  individual  numbers 
one  through  seven  refer  to  the  same  person  in  appendix  C  as  in  this  appendix, 
but  Individual  numbers  eight  through  thirty-one  in  appendix  C  correspond  to 
individual  numbers  nine  through  thirty-two  (respectively)  in  this  appendix. 
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RANKING  1 


CRITERION:  IMPORTANCE  TO  MISSION  WITH  ATTACK  ON  SUBMARINE 


INDIVIDUAL 

NUMBER 


DECISION  FUNCTION  GIVEN  RANK  OF: 


CS  VC 
AM  CP 
LP  AC 
CS  YC 
ME  VC 
CS  AC 
CS  VC 
CS  VC 
VC  CS 
CS  VC 
CS  DS 
CP  PC 
CP  LP 
CS  VC 
AC  LP 
LP  AC 
AC  LP 
ME  CS 
CP  CS 
LP  DS 
CP  VC 
VC  CS 
CS  LP 
CS  VC 
CS  CP 
PC  CP 
LP  AC 
CS  AC 
CS  VC 
AC  LP 
AC  LP 


TF  AC  AM 

MP  PC  CS 

CS  VC  TF 

CP  PC  TF 

CS  TF  AC 

DS  CP  TF 

AC  AM  CP 

TF  AM  AC 

TF  AC  DS 

TF  AC  CH 

LP  AC  TF 

EP  TF  AM 

AC  DS  MP 

TF  AM  LP 

PC  CP  TF 

CP  AM  CS 

DS  TF  AM 

VC  TF  AC 

VC  MP  TF 

AC  AM  CH 

CS  DS  LP 

AC  AM  TF 

DS  AC  TF 

AC  CH  LP 

TF  AC  LP 

MP  ME  CS 

DS  TF  AM 

DS  LP  CP 

CP  TF  AC 

DS  ME  CP 

TF  DS  AM 


CH  PC 
AP  TF 
DS  CP 
AM  AC 
DS  LP 
LP  PC 
PC  TF 
EP  AP 
LP  AM 
CP  EP 
VC  *  AM 
VC  CS 
TF  PC 
AC  ME 
ME  EP 
TF  DS 
CS  VC 
AM  LP 
AM  AC 
TF  EP 
AM  TF 
DS  EP 
AM  CP 
TF  AM 
AM  DS 
VC  TF 
CS  VC 
AP  EP 
AM  ME 
CS  TF 


8  9 


Lr 

DS  LP 
EP  AC 
AM  PC 
ME  DS 
AM  CP 
EP  AP 
DS  LP 
ME  MP 
PC  MP 
AM  ME 
MP  AP 
DS  AC 
CS  EP 
PC  CP 
AM  AP 
PC  VC 
CP  PC 
CP  AP 
DS  LP 
AP  ME 
AC  MP 
LP  CH 
PC  EP 
ME  DS 
AP  ME 
AC  DS 
MP  AP 
ME  TF 
PC  CP 
AM  MP 


10  11  12  13  14 


EP  CP 
DS  LP 
MP  ME 
EP  MP 
MP  PC 
VC  MP 
MP  AP 
PC  DS 
CP  EP 
AP  PC 
PC  EP 
LP  AP 
AM  AP 
MP  DS 
MP  DS 
ME  AP 
MP  EP 
EP  MP 
AP  PC 
CP  MP 
EP  PC 
PC  CP 
MP  ME 
EP  MP 
VC  EP 
LP  AP 
PC  CP 
MP  VC 
AP  DS 
AP  EP 


PC  CP  ME  CS  VC  MP 


MP  ME  AP 
VC  ME  CH 
EP  AP  CH 
AP  LP  CH 
AP  EP  CH 
AM  ME  CH 
EP  ME  CH 
LP  CP  CH 
AP  ME  CH 
MP  LP  DS 
CP  ME  CH 
MP  ME  CH 
VC  CH  ME 
AP  EP  CH 
CS  VC  CH 
MP  EP  CH 
AP  ME  CH 
PC  DS  CH 
EP  ME  CH 
PC  CS  VC 
ME  AP  CH 
AP  ME  MP 
VC  AP  CH 
AP  CP  PC 
MP  CH  PC 
EP  AM  CH 
EP  CH  ME 
AM  PC  CH 
MP  EP  CH 
PC  VC  CH 
EP  AP  CH 


RANKING  2 


CRITERION:  IMPORTANCE  TO  MISSION  WITH  TRACKING  OF  SUBMARINE  BUT  NO  ATTACK 


INDIVIDUAL 

NUMBER 

DECISION  FUNCTION  GIVEN  RANK  Or 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

- 1 - 

mm 

CP 

Ml 

CS 

TF 

AM  ' 

CH 

AP 

LP 

~sr 

|£fl 

LP 

2 

cs 

VC 

TF 

AM 

CH 

PC 

ME 

EP 

CP 

MP 

AP 

AC 

DS 

LP 

3 

AM 

CP 

MP 

PC 

CS 

AP 

TF 

CH 

EP 

ME 

VC 

AC 

DS 

LP 

4 

cs 

TF 

AM 

VC 

CH 

ME 

CP 

AP 

AC 

DS 

LP 

PC 

EP 

MP 

5 

CS 

VC 

CP 

PC 

TF 

AM 

CH 

ME 

EP 

MP 

AP 

AC 

LP 

DS 

6 

ME 

VC 

CS 

TF 

AM 

CP 

MP 

PC 

AP 

EP 

CH 

AC 

DS 

LP 

7 

CS 

CP 

TF 

CH 

PC 

EP 

AM 

AP 

VC 

MP 

AC 

DS 

LP 

ME 

8 

CS 

VC 

AC 

CH 

AM 

CP 

PC 

TF 

DS 

LP 

MP 

AP 

EP 

ME 

9 

CS 

VC 

TF 

AM 

AC 

CH 

EP 

AP 

ME 

PC 

MP 

DS 

LP 

CP 

10 

VC 

CS 

TF 

AM 

CH 

PC 

MP 

CP 

EP 

AP 

ME 

AC 

LP 

DS 

11 

CS 

VC 

CH 

CP 

EP 

AM 

ME 

AP 

PC 

MP 

TF 

LP 

OS 

AC 

12 

CH 

CS 

VC 

AM 

TF 

MP  -  PC 

EP 

ME 

CP 

LP 

DS 

AC 

DS 

13 

CP 

EP 

PC 

TF 

AM 

VC 

CS 

CH 

AP 

MP 

ME 

AC 

DS 

LP 

14 

CP 

MP 

PC 

VC 

CS 

EP 

AP 

ME 

AM 

TF 

CH 

LP 

AC 

DS 

15 

CH 

CS 

VC 

ME 

CP 

EP 

PC 

MP 

AP 

TF 

AM 

LP 

AC 

DS 

16 

CP 

ME 

EP 

PC 

AM 

MP 

AP 

AC 

TF 

CH 

YC 

CS 

LP 

DS 

17 

CP 

TF 

AM 

CH 

CS 

PC 

AP 

MP 

VC 

ME 

EP 

AC 

DS 

LP 

18 

TF 

AM 

CH 

CS 

VC 

CP 

PC 

MP 

EP 

AP 

ME 

AC 

LP 

DS 

19 

ME 

CP 

AP 

EP 

MP 

CS 

VC 

TF 

AM 

CH 

LP 

DS 

PC 

AC 

20 

CP 

CS 

VC 

MP 

TF 

AM 

AC 

DS 

CH 

LP 

AP 

PC 

EP 

ME 

21 

CP 

AM 

CH 

TF 

EP 

AP 

ME 

MP 

VC 

CS 

PC 

AC 

DS 

LP 

22 

CP 

VC 

CS 

CH 

AM 

TF 

MP 

EP 

PC 

ME 

AP 

AC 

LP 

DS 

23 

VC 

CS 

AM 

TF 

CH 

EP 

PC 

CP 

AP 

ME 

MP 

AC 

DS 

LP 

24 

CS 

CH 

AC 

AM 

TF 

ME 

AP 

PC 

CP 

MP 

EP 

VC 

LP 

DS 

25 

CS 

YC 

CH 

ME 

AM 

EP 

AP 

CP 

PC 

MP 

TF 

AC 

DS 

LP 

26 

CS 

CP 

TF 

AC 

AM 

CH 

ME 

PC 

EP 

MP 

YC 

AP 

LP 

OS 

27 

PC 

CP 

MP 

ME 

CS 

VC 

AP 

CH 

AM 

EP 

TF 

AC 

LP 

DS 

28 

CH 

AM 

TF 

CS 

VC 

MP 

AP 

PC 

CP 

EP 

ME 

AC 

DS 

LP 

29 

CS 

TF 

AM 

CH 

CP 

AP 

EP 

ME 

MP 

VC 

PC 

AC 

DS 

LP 

30 

CS 

VC 

CP 

TF 

CH 

ME 

PC 

AM 

AP 

MP 

EP 

AC 

LP 

DS 

31 

CH 

ME 

CP 

CS 

TF 

AM 

MP 

EP 

AP 

PC 

VC 

AC 

DS 

LP 

32 

TF 

AM 

ME 

PC 

CS 

VC 

CH 

EP 

MP 

CP 

AP 

LP 

AC 

DS 

RANKING  3 


CRITERION:  URGENCY  OF  THE  DECISION  FUNCTION  WHEN  IT  ARISES  IN  A  MISSION 


INDIVIDUAL 

NUMBER 


DECISION  FUNCTION  GIVEN  RANK  OF: 


2  3 


Hi: 


10  11  12  13  14 


V«l 

AP 

i  « 

LP 

»  M  I 

AM 

f  «v 

TF 

V/  v 

AC 

PC 

VC 

Ml 

ME 

DS 

CS 

CH 

CP 

•  M 

EP 

VI  • 

MP 

LP 

DS 

AP 

AM 

TF 

EP 

AC 

ME 

VC 

CS 

CP 

PC 

MP 

CH 

LP 

AC 

DS 

VC 

CS 

TF 

AM 

AP 

CP 

PC 

ME 

CH 

EP 

MP 

LP 

AC 

CH 

DS 

CP 

AM 

TF 

ME 

VC 

CS 

AP 

PC 

MP 

EP 

LP 

AC 

OS 

VC 

CS 

TF 

AM 

ME 

PC 

MP 

CP 

CH 

AP 

EP 

AC 

DS 

LP 

CS 

VC 

AP 

PC 

CP 

TF 

CH 

EP 

AM 

MP 

ME 

AC 

LP 

VC 

TF 

AM 

CS 

AP 

DS 

ME 

PC 

CP 

MP 

CH 

EP 

LP 

DS 

AC 

AM 

TF 

CP 

ME 

EP 

VC 

CS 

CH 

AP 

MP 

PC 

AC 

DS 

LP 

TF 

AM 

PC 

AP 

CP 

MP 

EP 

VC 

CS 

CH 

ME 

LP 

DS 

AM 

VC 

AC 

CH 

CS 

ME 

EP 

AP 

TF 

PC 

MP 

CP 

AC 

LP 

CS 

VC 

TF 

AM., 

DS 

CP 

PC 

EP 

MP 

AP 

ME 

CH 

AC 

LP 

DS 

AM 

VC 

CS 

TF 

CH 

AP 

ME 

MP 

EP 

PC 

CP 

LP 

AC 

TF 

DS 

AM 

AP 

VC 

CS 

EP 

ME 

CH 

PC 

MP 

CP 

CS 

VC 

TF 

AM 

LP 

AC 

ME 

CH 

DS 

PC 

CP 

EP 

MP 

AP 

LP 

AC 

TF 

DS 

ME 

PC 

AM 

AP 

EP 

MP 

CP 

CH 

VC 

CS 

LP 

DS 

AC 

CS 

AM 

TF 

YC 

CP 

ME 

EP 

CH 

PC 

AP 

MP 

AC 

LP 

DS 

PC 

ME 

TF 

CP 

AM 

CS 

VC 

CH 

AP 

MP 

EP 

ME 

DS 

LP 

AC 

AM 

TF 

VC 

CS 

AP 

MP 

EP 

CP 

CH 

PC 

CS 

VC 

AM 

AC 

LP 

PC 

TF 

CP 

MP 

ME 

EP 

AP 

DS 

CH 

DS 

LP 

ME 

AP 

EP 

AC 

PC 

AM 

CH 

TF 

CS 

VC 

CP 

MP 

AC 

DS 

LP 

TF 

AM 

VC 

CS 

ME 

CH 

EP 

AP 

CP 

MP 

PC 

TF 

AM 

LP 

AC 

VC 

CS 

DS 

ME 

EP 

AP 

PC 

CP 

CH 

MP 

LP 

AC 

TF 

AM 

DS 

AP 

ME 

VC 

CS 

CP 

MP 

EP 

PC 

CH 

LP 

DS 

AC 

VC 

CS 

CH 

TF 

EP 

AM 

AP 

ME 

CP 

MP 

PC 

AC 

LP 

DS 

VC 

AM 

CS 

TF 

ME 

CH 

PC 

EP 

AP 

MP 

CP 

ME 

DS 

AC 

LP 

AP 

TF 

YC 

CS 

AM 

EP 

CH 

MP 

CP 

PC 

AC 

AM 

TF 

CS 

VC 

LP 

DS 

MP 

AP 

PC 

CP 

EP 

CH 

ME 

AM 

TF 

AC 

DS 

LP 

CS 

YC 

ME 

MP 

EP 

AP 

CP 

CH 

PC 

VC 

CP 

DS 

AM 

ME 

PC 

AP 

AC 

TF 

CH 

CS 

CP 

EP 

MP 

ME 

PC 

CP 

MP 

AP 

EP 

AM 

LP 

DS 

CS 

VC 

AC 

TF 

CH 

AC 

LP 

AP 

ME 

CP 

AM 

TF 

PC 

EP 

CS 

VC 

DS 

CH 

MP 

RANKING  4 


CRITERION:  WORKLOAD  OF  TACCO  DURING  DECISION  FUNCTION 


INDIVIDUAL 

NUMBER 


DECISION  FUNCTION  GIVEN  RANK  OF: 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

KM 

PC 

AP 

TF 

LP 

AM 

AC 

CH 

VC 

ME 

DS 

CP 

EP 

MP 

PC 

CS 

LP 

AC 

DS 

AP 

EP 

AM 

TF 

CP 

CH 

ME 

MP 

CS 

VC 

PC 

DS 

LP 

AC 

CH 

VC 

CS 

TF 

AM 

CP 

PC 

ME 

AP 

MP 

EP 

CH 

CP 

ME 

LP 

AC 

AM 

DS 

TF 

PC 

MP 

AP 

EP 

VC 

CS 

ME 

LP 

DS 

AC 

PC 

VC 

CS 

CP 

MP 

CH 

AM 

TF 

AP 

EP 

CH 

AC 

DS 

TF 

PC 

LP 

AM 

CP 

EP 

MP 

AP 

CS 

VC 

ME 

LP 

AM 

AC 

VC 

TF 

DS 

CH 

CS 

AP 

ME 

PC 

EP 

MP 

CP 

AM 

AC 

TF 

DS 

LP 

VC 

CS 

CP 

EP 

ME 

PC 

AP 

MP 

CH 

CH 

ME 

LP 

AC 

DS 

AM 

TF 

AP 

MP 

CP 

EP 

PC 

CS 

VC 

CH 

DS 

LP 

AC 

AM 

VC 

AP 

ME 

TF 

CS 

EP 

PC 

MP 

CP 

CH 

AC 

TF 

AM 

CS 

VC 

-  PC 

LP 

MP 

EP 

AP 

DS 

ME 

CP 

AC 

LP 

DS 

CH 

AM 

TF 

CS 

VC 

ME 

AP 

MP 

EP 

PC 

CP 

LP 

ME 

CH 

TF 

DS 

AC 

MP 

CP 

AM 

PC 

EP 

AP 

VC 

CS 

ME 

CH 

TF 

AM 

PC 

CP 

MP 

AC 

EP 

AP 

CS 

VC 

LP 

DS 

AC 

LP 

PC 

ME 

TF 

AP 

DS 

AM 

EP 

MP 

CH 

VC 

CP 

CS 

LP 

DS 

AC 

AM 

TF 

CH 

ME 

CS 

VC 

CP 

PC 

EP 

MP 

AP 

CP 

PC 

ME 

EP 

CH 

AC 

TF 

MP 

CS 

VC 

AM 

LP 

DS 

AP 

ME 

TF 

AM 

AC 

LP 

AP 

EP 

MP 

CP 

DS 

PC 

CS 

VC 

CH 

CP 

CS 

VC 

TF 

AM 

AC 

AP 

EP 

ME 

MP 

PC 

LP 

DS 

CH 

DS 

LP 

ME 

AP 

EP 

AC 

PC 

AM 

CH 

TF 

CS 

VC 

MP 

CP 

LP 

DS 

AC 

TF 

AM 

VC 

CS 

CH 

ME 

AP 

EP 

PC 

CP 

MP 

CP 

AM 

TF 

AC 

CH 

LP 

DS 

M, 

EP 

AP 

PC 

VC 

CS 

MP 

CH 

DS 

LP 

AC 

TF 

AM 

CP 

AP 

MP 

ME 

EP 

PC 

VC 

CS 

DS 

LP 

AC 

CH 

AP 

EP 

YC 

CS 

TF 

CP 

MP 

AM 

PC 

ME 

TF 

CP 

AM 

CH 

PC 

ME 

CS 

EP 

VC 

LP 

MP 

AC 

DS 

AP 

CP 

EP 

AC 

LP 

DS 

TF 

ME 

CH 

AM 

VC 

CS 

MP 

AP 

PC 

AC 

LP 

TF 

CH 

AP 

AM 

CS 

VC 

MP 

PC 

CP 

DS 

EP 

ME 

ME 

AM 

CP 

EP 

TF 

AC 

CS 

VC 

PC 

LP 

DS 

MP 

CH 

AP 

CP 

PC 

ME 

TF 

LP 

DS 

AC 

CH 

MP 

AM 

AP 

EP 

VC 

CS 

ME 

CP 

AC 

CH 

TF 

AM 

PC 

MP 

AP 

EP 

LP 

VC 

CS 

DS 

AC 

ME 

CH 

LP 

TF 

DS 

AM 

PC 

AP 

EP 

MP 

CP 

VC 

CS 

APPENDIX  E 

ADDITIONAL  COMMENTS  FROM  MOFFETT  FIELD  TACCO  INTERVIEWS 


E.  ADDITIONAL  COMMENTS  FROM  MOFFETT  FIELD  TACCO  INTERVIEWS 


Upon  completion  of  the  MD  Scaling  interviews,  general  comments  were 
solicited  from  each  group  of  TACCOs  on  the  subjects  of  decision  aiding  and  P-3C 
hardware.  This  appendix  summarizes  the  comments  and  suggestions  received  from 
the  32  TACCOs  interviewed.  The  comments  concerning  general  aspects  of  the  P-3C 
are  reviewed  first,  after  which  the  more  specific  comments  are  presented. 


In  considering  the  overall  impact  of  automation  on  the  TACCO,  many  of 
the  interviewees  expressed  a  belief  that  much  of  it  unnecessarily  increased  the 
amount  and  complexity  of  their  work.  One  individual  commented,  for  example,  that 
he  felt  he  was  "working  for  the  computer,  instead  of  the  computer  working  for 
him."  Many  specific  examples  were  given  of  programs  possessing  excessive,  redun¬ 
dant  cueing,  and  functions  requiring  excessive  button-pushing.  It  was  thought 
that  such  problems  provide  a  roadblock  to  greater  acceptance  of  automation  by  the 
operational  community.  The  numerous  nonsubstantive  differences  between  similar 
programs  on  different  platforms  (e.g.  baseline  verus  update  I)  were  seen  as 
another  major  problem  with  previous  automation  efforts.  The  lack  of  consistant 
design  features  both  within  and  across  software  functions  was  generally  criti¬ 
cized.  The  interviewees  also  felt  that  there  were  many  programs,  displays,  etc. 
which  had  either  outlived  their  usefulness  or  had  never  been  useful,  but  which 
were  nonetheless  retained  in  the  system.  They  thought  those  'low-utility'  fea¬ 
tures  were  "cluttering"  the  displays,  consoles,  and  computer,  and  possibly  deter¬ 
ring  the  development  and  introduction  of  newer,  and  more  useful  system  features. 


The  majority  of  the  hardware-specific  comments  concern  the  TACCO  sta¬ 
tion  and/or  TACCO  functions,  as  this  was  the  primary  emphasis  of  the  interview. 
However,  a  few  also  concern  the  NAVCOM,  pilot,  and  sensor  operator  stations 
and/or  functions. 


E-l 


Hardware  related  suggestions/comments  pertaining  to  the  TACCO  station 

were: 

•  Provide  the  ability  for  TACCOs  to  monitor  both  NESTOR  and  one 
other  communications  system,  preferably  ICS.  When  using  NESTOR 
communications,  primarily  during  hand-off,  the  TACCO  is  unable  to 
monitor  any  other  communications,  including  ICS.  This  results  in 
the  TACCO  becoming  temporarily  removed  from  the  tactical  situation. 

•  Give  the  TACCO  the  ability  to  scuttle  all  types  of  sonobuoys  when 
they  are  no  longer  providing  target  information  to  the  ASW  crew, 
as  can  currently  be  done  with  CASS  sonobuoys. 

•  Develop  sonobuoys  having  variable  depth  settings  that  can  be 
changed  once  the  sonobuoy  is  deployed.  This  capability  would  be 
similar  to  that  available  in  the  CASS  buoys  by  which  the 
hydrophone  can  be  lowered. 

•  Provide  an  area  at  the  TACCO  station  which  can  be  used  for 
writing.  Currently,  the  TACCO  must  place  writing  material  over 
the  keyset  in  order  to  record  information. 

•  Provide  a  means  for  allowing  the  TACCO  to  see  sensor  operators' 
display  information,  without  having  to  leave  the  TACCO  station. 

•  Incorporate  a  color  display  at  the  TACCO  station  to  provide 
increased  comprehension  of  data  displayed. 

•  Relocate  the  TACCO' s  tableau  so  he  is  able  to  see  the  data  entry 
keyset  and  the  tableau  simultaneously. 

Hardware  related  suggestions/comments  pertaining  to  the  NAVCOM  station  were: 

•  Modify  the  high  speed  printer  such  that  the  printer  side  of  the 
paper  can  be  seen  without  having  to  move  it. 

•  Modify  the  high  speed  printer  (teletypewriter  (TTY)  feature)  so 
that  the  NAVCOM  can  see  what  he  is  sending  (TTY  Direct  Send) 
without  having  to  wait  for  two  lines  of  information. 

•  Have  a  clock  at  the  NAVCOM  staton  which  could  be  used  without 
recalling  tableaus  to  enter  time  information. 

t  Relocate  the  altimeter  at  the  NAVCOM  station  so  the  NAVCOM  does 
not  have  to  look  up  to  observe  it. 


•  NAVCOM  station  has  a  bad  draft  due  to  the  air  conditioning. 

•  Install  satellite  navigation  capability  for  use  by  the  NAVCOM 
operator. 

General  hardware  suggestions/comments  pertaining  to  the  overall  aircraft  were: 

•  All  the  seats  in  the  aircraft  get  very  uncomfortable.  It  would  be 
desirable  to  have  the  seats  recline  further  than  currently 
available. 

•  Vibrations  within  the  aircraft  are  uncomfortable. 

•  The  noise  generated  by  equipment,  especially  the  cooling  fans  on 
the  HF  radios  and  the  standby  gyro,  is  excessive.  Possibly 
install  acoustic  covering  on  the  bay  doors  to  reduce  the  noise 
level . 

•  Add-on  equipment  to  the  aircraft  operators'  stations  has  resulted 
in  less  than  optimal  human  factors  engineering  and  necessitates 
excessive  reaching.  Possibly  redesign  each  station. 

•  Investigate  the  use  of  plasma-type  displays  for  all  operators. 

Hardware  related  suggestions/comments  pertaining  to  the  sensor  operators  were: 

•  Install  displays  at  the  operator  stations  which  would  allow  the 
secondary  operator  to  backup  the  primary  operator  (e.g. ,  radar 
display  at  SS-I/II  or  acoustic  display  at  SS- III). 


Software  related  suggestions/comments  pertaining  to  the  P-3C  operating 
programs  (referring  to  both  Baseline  and  Update  I  aircraft)  were: 

•  Develop  an  extract  routine  which  would  search  all  stored  data, 
extract  it  and  format  it  into  the  RAINBOW  PURPLE  format.  This 
would  reduce  the  amount  of  time  the  TACCO  spends  writing  during 
the  tactical  phase. 

•  In  the  baseline,  develop  the  ability  to  reserve  RF  and  sonobuoy 
types  for  future  patterns,  similar  to  the  capability  currently  in 
the  Update  I  version. 

•  Shorten  the  cueing  sequence  on  the  store  selection  program. 

•  Tableau  2  on  the  Baseline  aircraft  is  ineffective  and  provides  no 
useful  information. 
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In  conjunction  with  weapon  fly-to-points,  develop  and  display 
probability-of-kill  contours  for  the  selected  weapons.  This  could 
be  used  to  allow  the  TACCO  to  release  a  weapon  optimally  if  the 
pilot  is  unable  to  capture  the  fly-to-point. 

Provide  the  ability  to  transmit  written  information  (formatted  or 
free  format)  to  other  aircraft  during  swaps.  Types  of  information 
desired  include  NOR,  buoys  in  use,  life  and  depth  settings,  layer 
depth,  and  target  data. 

Have  a  computerized  preflight  of  the  aircraft. 

Incorporate  some  of  the  HP-67  programs  into  the  aircraft  operating 
programs,  specifically  the  HOWGOZIT,  PIM,  LOP,  NAVPREFLIGHT,  and 
LLOYDS  MIRROR  programs. 

In  the  development  of  a  weapon  fly-to-point,  incorporate  the  time- 
late  factor  involved  with  the  sensor  used  to  determine  the  fix. 

Eliminate  all  differences  between  similar  functions  of  the  Baseline 
and  Update  I  operating  programs. 

Have  an  alert  notify  the  pilot  that  he  has  something  on  his 
display  (e.g.,  fly-to-point)  to  which  he  must  respond.  This  would 
reduce  the  ICS  between  the  pilot  and  TACCO. 

On  the  PD  IP  tape,  include  acoustic  information  and  ESM  information 
on  the  targets-of- interest  which  can  be  displayed  to  the  TACCO. 

Have  a  recommended  altitude  profile  which  is  displayed  to  the 
pilot  and  updated  according  to  the  tactical  situation. 

Provide  target  course  readouts  when  amplifying  the  predict 
symbol . 

In  the  generation  of  weapon  fly-to-points,  utilize  course 
information  on  the  submarine,  and  generate  steering  commands  to 
have  the  aircraft  fly  up  the  course  of  the  submarine  for  the 
weapon  release. 

Use  real-time  environmental  information  for  the  development  of 
sensor  patterns. 

Incorporate  a  time-out  feature  on  sonobuoy  symbols  which  would 
remove  the  symbol  when  its  effective  lifetime  has  been  exceeded 
and  no  RF  from  that  channel  is  detected. 
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•  When  selecting  and  setting  weapons,  have  the  computer  queue  two 
weapons  in  the  event  that  a  weapon  station  is  blocked  at  time 
of  release. 

•  Provide  the  capability  to  expand  DIFAR  and  ACTIVE  buoy  simultan¬ 
eously  with  only  one  cueing  sequence. 

•  Incorporate  a  mission  phase  checklist  for  display  to  the  TACCO. 

e  Provide  the  TACCO  with  an  indication  of  the  type  of  bearing 
information  (ALI,  TRACKER)  being  displayed. 

•  Display  task-force-related  areas  of  interest  (submarine  areas, 
standoff  areas)  which  will  be  updated  based  upon  Task  Force  PM. 

•  Display  HARPOON  kill  radius  that  moves  with  aircraft. 

•  Incorporate  multi-target  Doppler  tracking  algorithms. 

•  Improve  the  CZ  displays,  possibly  to  show  only  areas  of  proba¬ 
bility  from  intercept  of  multiple  CZ  ranges. 

e  Maintain  past  history  on  DIFAR  bearings  for  display  to  TACCO. 

«  Have  the  auto  track  function  displayed  along  with  probability 

area  contours. 

•  Automatically  extract  NAVPARAMETERS  Tableau  on  the  HSP  whenever 
specified  events  occur  (e.g.,  buoys  release)  or  time  periods  expire. 

•  Incorporate  geographical  map  overlays  for  display  to  TACCO  and 
SS- III  operator. 


It  should  be  noted  that  some  of  these  comments  were  voiced  by  only  one 
or  two  personnel.  However,  many  of  them  were  repeated  by  many  individuals.  The 
most  frequently  repreated  comments  were  those  regarding  store  management/reuse 
RF  in  the  Baseline  P-3,  the  ability  to  have  the  computer  generate  a  PURPLE 
message,  incorporation  of  existing  HP-67  programs  into  the  operational  programs, 
improved  weapons  fly-to-points,  ability  to  transmit  text  material  in  a  swap,  and 
utilization  of  environmental  information  for  pattern  construction. 
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F.  UNFOLDING  ANALYSIS  AND  PRIORITY  MAPPING 


In  a  multidimensional  model  of  a  psychological  domain,  such  as  the  MDS 
model  of  TACCOs  perceptions  of  ASW  decision-making  constructed  in  the  previous 
section,  each  item  in  the  domain  has  a  unique  projection  or  coordinate  on  each 
of  the  dimensional  axes.  Each  dimension  is  therefore  a  scale  which  measures 
some  salient  perceptual  attribute  of  the  domain.  Together,  all  these  scales 
measure  all  the  independent  features  needed  to  represent  the  perceived  interre¬ 
lationships  among  the  items  in  the  domain.  In  the  ASW  decision-making  case,  the 
three  dimensions  (as  shown  in  Table  3-1)  provide  numeric  measures  of  all  the 
important  features  by  which  experienced  TACCOs  perceive  ASW  decisions. 

Other  characteristics  or  features  of  the  items  scaled  must  therefore  be 
combinations  of  the  features  represented  by  the  exclusive  and  exhaustive  dimen¬ 
sions  in  the  multidimensional  space.  Such  derivative  features  may  have  dif¬ 
fering  properties  as  measurement  scales,  and  may  therefore  have  different 
representations  within  the  multidimensional  space.  The  feature  may  act  as  a 
categorical  scale,  in  which  case  it  will  be  represented  as  a  partition  of  the 
multidimensional  space  into  unordered  regions.  The  feature  may  act  as  an  ordi¬ 
nal  scale,  in  which  case  it  will  be  represented  as  a  graded  or  ordered  partition 
of  the  multidimensional  space.  The  feature  may  even  act  as  a  ratio  or  interval 
scale  (similar  to  the  dimensions  themselves),  in  which  case  it  will  be  repre¬ 
sented  as  a  directed  vector  or  curve  passing  through  the  multidimensional  space. 

Various  techniques  can  be  used  to  construct  representations  of  deriva¬ 
tive  features  in  an  existing  multidimensional  model.  Little  can  be  done  to 
represent  categorical  derivative  features  in  the  multidimensional  model.  More 
can  be  done  to  construct  a  representation  of  ordinal  or  ratio-scale  derivative 
features,  through  the  use  of  a  powerful  technique  known  as  Unfolding  Analysis. 
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This  technique  was  first  proposed  by  Coombs  (1950)  and  later  generalized  by 
Bennett  and  Hays  (1960).  It  is  based  on  the  concept  that  the  general  form  of 
the  representation  of  a  derivative  feature  in  a  predefined  multi  dimensional 
space  can  be  determined  by  the  comparison  of  the  trial  representations  produced 
by  a  series  of  different  models  of  increasing  complexity. 

Unfolding  Analysis  postulates  that  there  are  two  basic  ways  in  which 
derivative  features  can  be  expressed  as  a  combination  of  the  dimensions  in  the 
multidimensional  space—  either  as  a  monotonic  combination  of  all  the  dimensions, 
or  as  a  non-monotonic  combination  of  the  dimensions.  When  the  relationship  is 
monotonic,  increased  values  on  the  dimensions  will  always  yield  monotonic 
changes  in  the  derivative  scale.  When  the  relationship  is  non-monotonic,  then 
increased  values  on  a  dimension  will  yield  monotonic  changes  in  the  derivative 
scale  only  up  to  some  point,  beyond  which  the  tonicity  of  the  derivative  scale 
will  reverse.  Thus,  in  a  non-monotonic  combination,  there  will  be  a  point  on  each 
dimension  such  that  the  derivative  scale  will  increase  on  one  side  of  it  and 
will  decrease  on  the  other  side  of  it  as  values  on  the  dimension  increase. 

While  there  are  infinitely  many  monotonic  and  non-montonic  ways  of  com¬ 
bining  dimensions,  Unfolding  Analysis  restricts  itself  to  four  specific  formulae 
wnich  have  simple  mathematical  forms  and  direct  psychological  interpretations. 
Although  the  analysis  can  be  performed  on  either  ordinal  or  ratio  scale  features, 
the  discussion  here  will  be  restricted  to  ordinal  scales,  because  the  data 
collected  at  Moffett  Field  used  ordinal  measurement  scales. 

In  Unfolding  Analysis,  the  four  different  models  are  sequentially  con¬ 
sidered  in  terms  of  their  ability  to  represent  a  given  ordinal  scale  in  a  given 
multidimensional  space.  Normally,  the  multidimensional  space  is  the  result  of 
an  MDS  analysis  and  the  ordinal  scale  is  a  ranking  (or  set  of  rankings)  performed 
by  one  or  more  individuals  according  to  some  criteria.  The  Unfolding  Analysis 
then  can  be  thought  of  as  an  attempt  to  determine  the  best  representation  of  the 
ranking  criteria  as  a  function  of  the  dimensions  in  the  MDS  solution. 
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Once  the  form  of  the  representation  of  the  criterion  has  been  selected, 
regression- like  techniques  can  be  used  to  precisely  define  an  equation  which 
models  it  as  a  function  of  the  dimensions  in  the  space.  Carroll  and  Chang  (1967) 
have  developed  a  set  of  computational  procedures  for  performing  these  regression 
analyses.  Because  their  method  was  designed  to  relate  MDS  solutions  to  data  on 
the  preference  or  desirability  of  the  items  in  the  multidimensional  space,  they 
have  termed  it  "Preference  Mapping."  In  the  analysis  performed  here,  their  com¬ 
putational  approach  is  used,  but  instead  of  modeling  data  on  preference  it  is 
used  to  model  data  on  perceived  priority  of  decision  functions.  Thus,  this 
application  of  the  model  is  more  appropriately  termed  Priority  Mapping,  since  it 
constructs  a  mathematical  "map"  of  the  TACCOs  intuitive  prioritization  of  the 
decision  functions. 

F.l  FOUR  MODELS  OF  PRIORITY 

The  unfolding  analysis  and  priority  mapping  procedures  each  employ  four 
models  of  priority.  These  models  are  placed  into  a  strict  hierarchy,  as  each 
model  can  be  shown  to  be  a  special  case  of  the  next  higher  model.  The  highest 
model  in  the  hierarchy,  then,  subsumes  all  of  the  lower  three.  Because  the  data 
to  be  analyzed  here  are  ordinal,  the  models  will  be  explained  in  terms  of  ordi¬ 
nal  representations. 

An  ordinal  scale  in  a  multidimensional  space  divides  the  space  into 
graded  regions,  such  that  a  point  in  any  one  region  has  a  different  rank  than  a 
point  in  any  other,  but  has  the  same  rank  as  another  point  in  the  same  region. 

Thus  the  multidimensional  space  is  broken  into  isopriority  regions ,  which  are 
separated  by  isopriority  contours.  The  four  models  can  best  be  described  by  the 
different  kinds  of  isopriority  contour  structures  they  allow.  The  kinds  of  iso¬ 
priority  contours  permitted  by  the  four  models  are  depicted  in  Figure  F-l.  In  the 
following  paragraphs,  the  differences  between  these  four  representations  are  dis¬ 
cussed  in  detail.  For  simplicity,  Figure  F-l  and  the  following  discussion  are 
expressed  in  terms  of  a  two-dimensional  space.  The  generalization  of  the  approach 
to  three  or  more  dimensions  is  straightforward  and  not  explicitly  presented. 
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F.1.1  The  Vector  Model  I 

The  first  model,  and  the  lowest  on  the  hierarchy,  is  the  only  monotonic 
model.  In  fact,  it  is  not  only  monotonic,  it  is  linear.  The  model  assumes  that  3 

each  dimension  contributes  to  priority  in  a  linear  fashion,  so  that  the  priori- 
tization  of  items  is  defined  by  their  projections  onto  a  linear  vector  in  the  ; 

multidimensional  space.  The  isopriority  contours  in  two  dimensions  are  therefore 
lines  which  are  perpendicular  to  this  vector,  as  shown  in  the  upper  right  por¬ 
tion  of  Figure  F-l.  Because  this  model  defines  priority  in  terms  of  a  vector,  \ 

it  is  termed  the  vector  model.  The  vector  priority  model  is  quite  similar  to 
the  concept  of  multiattribute  utility.  Each  attribute  (dimension)  contributes 
to  the  overall  score  in  a  linear  way,  with  the  relative  contribution  of  each  , 

dimension  defined  by  a  coefficient  assigned  to  it.  In  the  vector  model  (Figure 
F-l),  the  coefficient  assigned  to  a  dimension  is  the  cosine  of  the  priority  vec¬ 
tor  with  regard  to  that  dimensional  axis.  The  vector  model  can  be  interpreted 
as  a  "more  of  a  good  thing"  model,  because  it  assumes  that  whatever  a  dimension 
represents,  more  of  it  will  always  contribute  to  a  higher  priority.  There  can 


never  be  "too  much"  of  any  dimension  in  a  vector  model  representation. 

F.1.2  The  Unweighted  Distance  Model 

The  remaining  three  models  are  non-monotonic  and  define  priority  in 
terms  of  distance  in  the  multidimensional  space.  They  utilize  the  notion  that 
there  is  an  "ideal  point"  somewhere  in  the  multidimensional  space,  such  that  the 
closer  a  decision  is  to  the  ideal  point  the  higher  its  priority.  Alternatively, 
the  ideal  point  can  be  an  anti-ideal  point,  such  that  the  further  a  decision  is 
from  it,  the  higher  the  priority  of  the  decision.  The  simplest  of  the  distance 
models  is  termed  the  unweighted  distance  model  or  just  the  distance  model.  It 
models  the  prioritization  of  the  items  in  the  space  strictly  according  to  their 
distance  from  the  ideal  point,  as  defined  by  the  distance  metric  constructed 
for  the  space  with  the  MDS  analysis.  Normally  this  metric  will  give  the 
Euclidean  definition  of  distance,  and  the  isopriority  contours  will  therefore 
appear  (in  two  dimensions)  as  concentric  circles  around  the  ideal  point,  as 


shown  in  the  upper  right  of  Figure  F-l.  The  vector  model  can  be  seen  as  a  spe¬ 
cial  case  of  the  distance  model  in  which  the  ideal  point  is  located  infinitely 
far  from  the  origin  along  the  priority  vector.  As  the  ideal  point  moves  toward 
infinity,  the  concentric  isopriority  circles  asymptote  to  lines  which  are  per¬ 
pendicular  to  the  priority  vector. 

The  distance  model  assumes  that  distance  alone  is  the  basis  for  priority 
and  therefore  that  each  dimension  has  a  equal  contribution  to  priority.  This  is 
because  each  dimension  contributes  equally  to  the  distance  of  any  one  decision 
from  any  other,  or  from  the  ideal  point.  Because  distance  is  a  quadratic  func¬ 
tion  of  the  dimensions  of  a  space  the  distance  model  and  all  other  non-monotic 
models  in  the  hierarchy  are  based  on  a  quadratic  representation  of  priority. 

The  vector  model,  in  comparison,  is  based  on  a  linear  representation. 


F.1.3  The  Weighted  Distance  Model 

A  more  general  representation  of  priority  in  terms  of  distance  allows 
the  distance  between  the  ideal  point  and  a  decision  on  certain  dimensions  to 
contribute  more  toward  the  decision's  priority  than  the  distance  on  other 
dimensions.  To  model  this,  each  dimension  is  assigned  a  weight  or  saliency. 

The  weights  are  used  in  computing  the  distance  between  the  ideal  point  and  each 
decision  on  each  dimension,  and  therefore  affect  the  decision's  priority.  This 
model,  termed  the  weighted  distance  model,  allows  isopriority  contours  to  assume 
(in  two  dimensions)  the  shape  of  an  ellipse  (or  any  other  conic  curve)  parallel 
to  the  dimensional  axes,  as  shown  in  the  lower  left  of  Figure  F-l.  The 
unweighted  distance  model  is  obviously  a  special  case  of  the  weighted  model,  in 
which  all  the  dimensional  weights  are  1.0. 

F.1.4  The  Generalized  Distance  Model 

8oth  the  weighted  and  unweighted  distance  models  assume  that  each 
dimension  contributes  independently  to  priority.  It  is  quite  possible,  however, 
that  there  is  interaction  among  some  or  even  all  of  the  dimensions  in  constituting 
the  concept  of  priority.  A  model  which  permits  such  interactions  among  dimensions 
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can  be  constructed  by  allowing  the  axes  to  be  rotated  in  the  multidimensional 
space  before  the  distances  are  computed.  The  isopriority  contours  in  this  kind 
of  model  would  still  be  ellipses  or  other  conics  (in  two  dimensions)  as  in  the 
weighted  distance  model,  but  in  this  case  they  can  assume  any  orientation  in  the 
space.  This  is  shown  in  the  lower  right  of  Figure  F-l.  This  model,  termed  the 
generalised  distance  model  allows  each  dimension  to  contribute  independently  and 
in  combination  with  each  of  the  others  to  the  priority  of  each  decision  function. 


F.2  SELECTION  OF  THE  OPTIMAL  REPRESENTATION 

In  an  actual  application  of  the  Priority  Mapping  precedure,  trial 
representations  of  the  (ranking)  data  are  constructed  for  all  four  of  the 
unfolding  analysis  models.  Each  of  these  representations  will  produce  a  dif¬ 
ferent  ranking  of  the  stimuli,  and  these  'model'  rankings  can  be  statistically 
compared  to  original  data  to  determine  the  appropriateness  of  each  model.  The 
,  simplest  and  most  reasonable  measure  that  can  be  used  for  such  a  comparison  is 

the  correlation  between  the  model  ranking  and  that  of  the  data.  The 
correlation,  when  squared,  becomes  the  coefficient  of  variation  statistic  (r2) 

;  which  indicates  the  proportion  of  the  variance  in  the  data  that  is  accounted  for 

by  the  model.  An  F  statistic  is  then  computed  from  the  value  of  r2  and  used  to 
test  the  null  hypotheses  that  there  is  no  underlying  correspondence  between  the 

model  and  the  data  (i.e.  that  the  value  of  r2  is  strictly  a  chance  result). 

■I 

1  Those  model  representations  for  which  the  null  hypothesis  can  not  be 

[’  rejected  at  a  reasonable  level  of  confidence  (say  95%  or  99%)  can  be  eliminated 

f : 

from  further  consideration.  If  this  is  the  case  for  all  of  the  models,  it  must 
be  assumed  that  none  of  them  is  appropriate  for  the  data,  and  that  Priority 
Mapping  can  not  be  used  to  prioritize  the  stimuli  in  terms  of  the  specified 


of  r2  --  the  proportion  of  variance  accounted  for.  Each  model  in  the  hierarchy 
will  normally  account  for  more  variance  than  any  of  the  lower  models,  because 
each  (higher)  model  in  the  hierarchy  attempts  to  fit  the  date  to  a  more  complex 
equation  than  all  the  lower  ones,  and  thus  contains  more  parameters.  Other 
things  being  equal,  a  model  with  more  parameters  will  always  explain  more  of  the 
variance  in  a  data  set  than  a  model  with  fewer  parameters,  so  it  is  reasonable 
to  expect  the  values  of  r^  to  increase  as  the  hierarchy  is  traversed  from  bottom 
to  top.  The  key  question  is  whether  or  not  the  increase  in  r^  of  one  model  over 
another  is  significant  when  considered  against  the  increase  in  the  number  of 
parameters. 

This  problem  requires  a  statistical  comparison  among  the  rankings  pro¬ 
duced  by  the  four  models.  An  F  statistic  can  be  computed  from  the  x2  values  of 
each  pair  of  models  to  test  the  null  hypothesis  that  the  increase  in  x2  of  one 
model  over  the  other  is  only  a  chance  result  produced  by  the  increase  in  the 
number  of  parameters.  A  rejection  of  this  null  hypothesis  for  a  given  pair 
indicates  the  model  that  is  higher  in  the  hierarchy  (the  more  powerful  model) 
provides  a  significantly  better  representation  of  the  data  than  does  the  model 
that  is  lower  in  the  hierarchy.  A  failure  to  reject  the  null  hypothesis  for  a 
given  pair  indicates  that  there  is  not  a  significant  difference  in  the  explana¬ 
tory  power  of  the  two  models,  and  that  the  model  lower  in  the  hierarchy  should 
be  preferred  because  it  is  simpler.  By  comparing  all  pairs  of  significant 
models  in  this  manner,  the  optimal  representation  can  be  chosen. 


APPENDIX  G 

UNFOLDING  ANALYSIS  AND  PRIORITY  MAPPING  RESULTS 


0 


G.  UNFOLDING  ANLYSIS  AND  PRIORITY  MAPPING  RESULTS 


This  appendix  presents  the  details  of  the  Unfolding  Analysis  and 
Priority  Mapping  procedures  as  applied  to  the  two  sets  of  importance  rankings 
obtained  at  Moffett  Field.  Section  G.i  describes  the  analyses  of  the  importance 
rankings  in  attack  missions,  and  Section  G.2  describes  the  analyses  of  the 
importance  rankings  in  surveillance  missions. 

G.I  PRIORITIZATION  OF  IMPORTANCE  RANKINGS  FOR  ATTACK  MISSIONS 

Using  the  PREFMAP  program,  the  unfolding  analysis  was  begun  by 
constructing  trial  representations  of  this  ranking  criterion  using  each  of  the 
four  hierarchial  models  discussed  in  Appendix  F.  These  representations  were 
statistically  compared  to  the  rankings  made  by  the  TACCOs  interviewed  at  Moffett 
Field  to  determine  the  appropriateness  of  each  model.  The  comparisons  were  made 
using  the  r  and  r2  statistics,  as  described  in  Appendix  F.  The  significance  of 
the  comparisons  against  a  null  hypothesis  that  there  is  no  underlying  correspon¬ 
dence  between  the  model  and  the  data  (i.e.  that  the  value  of  r2  is  strictly  a 
chance  result)  was  then  tested  using  an  F  statistic.  The  values  of  r2  ,  the  F 
statistic,  and  the  resulting  rejection  level  for  the  null  hypothesis  are  given 
in  Table  G-l. 


Table  G-1.  Significance  Of  Results  For  Mission  With  Attack 


MODEL 

NUMBER 

MODEL  FORM 

100-r2  (%  VARIANCE 
ACCOUNTED  FORI 

F  STATISTIC 

REJECTION  LEVEL 
FOR  NULL  HYPOTHESIS 
FOR  CHANCE  RESULT 

1 

GENERALIZED 

99.4 

77.1 

<.01 

2 

WEIGHTED  DISTANCE 

94.6 

21.1 

<.01 

3 

UNWEIGHTED  DISTANCE 

91.3 

23.9 

<.01 

4 

VECTOR 

73.4 

9.28 

<.01 

All  four  models  produced  a  significant  {i.e.  non-chance)  representation  of  the 
data,  indicating  that  each  model  was  capable  of  providing  a  reasonable  represen¬ 
tation  of  the  TACCO  rankings.  The  next  step  in  the  an lysis  was  to  determine 
which  produced  the  beat  representation. 

As  described  in  Appendix  F,  comparisons  were  made  between  all  pairs  of 
model  representations  using  an  F-statistic.  Each  comparison  considers  the 
increase  in  r^  provided  by  one  model  over  another  against  the  greater  number  of 
parameters  needed  to  achieve  that  increase  in  r^*  These  comparisons  are  sum¬ 
marized  in  Table  G-2. 


Table  G*2.  Comparison  Of  Models  For  Mission  With  Attack 


MODEL 


hypothesis  of  no  difference 


MODEL 

•DENOTES  SIGNIFICANT  DIFFERENCE 


As  seen  in  Table  G-2  the  generalized  distance  model  (Model  1)  is  signi¬ 
ficantly  more  powerful  than  all  of  the  other  models,  with  a  confidence  of  97.5%. 
Therefore,  it  was  determined  to  be  the  most  appropriate  of  the  four  models  for 
representing  the  rankings  of  the  TACCOs  according  to  this  criterion. 

Having  identified  the  best  form  of  the  function  representing  the  cri¬ 
terion  "importance  in  a  mission  with  attack"  as  that  specified  by  the 
generalized  distance  model,  a  precise  priority  function  was  "mapped",  again 
using  the  PREFMAP  program.  The  importance  of  priority  of  a  decision  function  j 
in  a  mission  with  attack  (PwA(dj))  was  f°und  To  be  given  by: 

PWA(dj)  =  4.87X?1  -  1.78X?2  +  .682X?3  -  .776XjlXj2  -  2.31XjlXJ3 
-3.71Xj2  j3  +  *299Xj!  +  .541Xj2  +  .858Xj3  -  .301 


where  Xji  is  the  location  or  coordinate  of  dj  on  dimension  1, 

Xj2  is  the  coordinate  of  dj  on  dimension  2,  and 
Xj3  is  the  coordinate  of  dj  on  dimension  3 

The  values  of  the  Xjj,  Xj3,  and  Xj3  were  given  in  Table  3-1.  The  priority 
values  for  the  fourteen  decision  functions  generated  by  this  function  were  given 
in  Table  4-1. 

G.2  PRIORITIZATION  OF  IMPORTANCE  RANKINGS  FOR  SURVEILLANCE  MISSIONS 

An  identical  analysis  was  conducted  for  the  TACCO  rankings  of  decision 
function  importance  in  a  mission  without  attack  —  one  where  the  goal  is  only 
surveillance  of  the  submarine.  PREFMAP  was  used  to  construct  trial  represen¬ 
tations  of  the  aggregated  TACCO  rankings  according  to  this  criterion.  These 
rankings  were  then  compared  with  the  TACCO  rankings  to  determine  which  of  the 
four  models  produced  significant  representations  of  the  TACCO  data.  These 


comparisons,  using  the  correlation  between  rankings  (r),  the  proportion  of 
variance  accounted  for  (r2),  and  the  statistic  test  (F)  for  a  null  hypothesis  of 


Table  G-3.  Significance  of  Results  For  Mission  Without  Attack 


pm 

MODEL  FORM 

100-r2  1%  OF  VARIANCE 
ACCOUNTED  FORI 

F  STATISTIC 

REJECTION  LEVEL 

FOR  NULL  HYPOTHESIS 
OF  CHANCE  RESULT 

GENERALIZED 

99.0 

9S.8 

<.01 

2 

WEIGHTED  DISTANCE 

96.0 

28.4 

<.01 

3 

UNWEIGHTED  DISTANCE 

92.0 

26.4 

<.01 

4 

VECTOR 

83.0 

16.4 

.025 

As  with  the  previous  criterion  (importance  to  a  mission  with  attack),  the  null 
hypothesis  can  be  rejeced  for  all  four  models,  indicating  that  each  of  them  pro¬ 
vides  a  significant  representation  of  the  aggregated  TACCO  rankings. 

The  four  models  were  then  compared  against  each  other  to  determine  which 
of  them  provided  the  best  representation  of  the  TACCO  rankings.  The  F  statistic 
was  used  to  compare  each  pair  of  models,  as  described  in  Appendix  F,  testing  a 
null  hypothesis  that  the  increase  in  the  proportion  of  the  variance  accounted 
for  by  the  more  powerful  of  the  two  models  was  merely  a  chance  result  caused  by 
its  increased  number  of  parameters.  These  comparisons  are  shown  in  Table  G-4. 
There  are  more  nonsignificant  comparisons  in  Table  G-4  than  there  were  in  Table 
G-2  (the  comparisons  for  the  previous  criterion).  In  particular,  there  were  no 
significant  differences  among  the  less  general  models.  However  in  this  case  (as 
in  the  previous  one)  the  general  distance  model  was  found  to  be  significantly 
better  than  any  of  the  other  models  in  representing  the  TACCO  rankings  according 
to  this  criterion.  Therefore,  it  was  determined  to  be  the  most  appropriate  of 
the  four  models  for  this  criterion. 

Having  determined  that  the  general  distance  model  provides  the  best 
functional  representation  of  the  TACCO  rankings  as  a  combination  of  the  three 
dimensions  identified  in  the  MDS  analysis,  PREFMAP  was  then  used  again  to  map 
the  precise  priority  function.  Using  this  program,  the  importance  or  priority 


Table  G-4.  Comparison  Of  Models  For  Mission  Without  Attack 


MOOEL 


F  STATISTIC 

REJECTION  LEVEL  FOR  NULL 
HYPOTHESIS  OF  NO  DIFFERENCE 


MODEL 

*OENOTES  INSIGNIFICANT  DIFFERENCE 


of  a  decision  function  j  on  a  mission  without  attack  (PwOA(dj))  was  found  to  be 
given  by: 

pW0A<dj)  *  3.5 9X? x  -  3. 48X^2  ♦  7.85x23  -  S^X^X^  -  1.40Xj2Xj3 
+5.70Xj2  j3  -  • 1 5 7 X j 2  +  2.0 7Xj2  +  .484Xj3  +  .207 


where  Xj^  is  the  coordinate  of  decision  function  j  on  dimension  i.  The  values  of 
the  Xji  were  given  in  Table  3-1.  The  priority  values  for  the  fourteen  decision 
functions  generated  by  this  function  are  given  in  Table  4-2. 
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